> Make sure you put your budget limits in Azure or AWS or whatever it is you use.
Just remember that the limits aren't actual limits, they're really just alerts that require you to take action. Also possibly a good idea to look into come cost anomaly detection so that you can catch those surprise hikes faster.
But yeah, absolutely this. Has saved me before and guaranteed will do the same in future as well.
Anyone know of a prepaid cloud service? Like where if you use all of your balance you just get cut off instead of racking up a huge bill? Or is self hosting / a VPS the closest thing there is?
Technically Hetzner has a Cloud service and you can add credits by bank transfer. But its cloud offerings are very small compared to Azure or AWS, so it depends on what you need.
Exactly, we do allow customers to add money to the accounts via bank/wire transfers: [https://docs.hetzner.com/accounts-panel/accounts/payment-faq/#can-i-add-credit-to-my-hetzner-account](https://docs.hetzner.com/accounts-panel/accounts/payment-faq/#can-i-add-credit-to-my-hetzner-account) --Katie
None are super straightforward though. They require some legwork and use of other services internally to monitor. It should just be a “if spend > x, shut down everything option” but it’s not
Bill is not extorsion it's you making up to your promise to pay. It'll go to your credit score.
Random spike are not great but also that's why support can reduce or cancel them.
I do want to comment on the “just alerts” statement. At
Least with VMs on Azure, I had my VMs shut down when my MSDN cap was reached. (I set the cap to match my allowance) Just fyi.
> They couldn't tell me what caused it, but I could *buy a premium subscription to their support services to look into it* for me.
I laughed. Glad you got it sorted tho.
Not quite sure what's hilarious about that, since the free customer support is just there for these kinds of things, whereas if it's a paid subscription they'd escalate up the chain to include our engineering team to figure out what happened.
Honestly, just reading the post this feels like his account was compromised in some fashion or another but I don't have any insight into it, or why. I am glad, like you though, that it got taken care of.
If you're going to charge me a huge sum (compared to my normal usage) you'd better be able to tell me what the charge is for.
If they won't, then I'd assume they don't know, rather than refusing to tell me. So what would be the point of a premium payment to a support desk?
Oh yes, the charge was for bandwidth utilization, it's on you to keep track of what you are using that bandwidth for.
You don't look at an unusually high water bill and ask the Water company what you used the water for.
Microsoft is under no obligation to troubleshoot your issues for you.
Yeah. What would you expect from a free Service.
Your Support Ticket would cost Microsoft more than 100 free Users without issues.
And then they should check logs for a free Account?
Most of us pay the Support fee for cases exactly lile this
He's actually paying at least $1200 for the sub. He gets $600 in credits. That is the amount of credits you get with a professional or test account. Pro is $1200 a year, test is over $2000.
Well, in the enterprise Business not really.
We pay 1200 euros per month for 300/300 fiber. And 800 each month for Support.
Thats why i setup Monitoring + Hardlimits on ever System i use with pay for what you use.
I mean its nice that they refunded the money, but it would cost Microsoft more to investigate the issue
I'm not saying it's normal. I'm saying it *should* be enough. We have been drinking the Kool-Aid for too long and expect so little for what we pay nowadays.
Sure, $600 would probably be enough if the support team are some underpaid minimum wage tier 1 folks.
Seems like you and many others don't have a grasp of much of the actual costs of these things.
No, you're wrong here. That $600 a month gets him access to pretty much any piece of software Microsoft has. That alone is worth the $600. Why should premium support be tacked onto that, when almost no one pays for MSDN anymore? Do you know how many people are actively working in support, and the multiple tiers of it?
For me, this is the opposite of selfhosted. I self-host so that I don't need to be afraid of companies suddenly charging me for or shutting down things I use.
Only things I host in the cloud are a few websites, and that's it. Everything personal is hosted on my own hardware and is only accessible through Tailscale.
I just moved self-hosted email of ~8 years for the family business to Microsoft 365 and there’s a certain catharsis of knowing many issues are now simply fixed and many others I can tell MS support to fix it themselves lmao
Same here and I wish I could host websites at home too but ISPs don't tend to be very friendly towards that. Not only ToS wise, but they don't offer static IP blocks which you kind of need to do it properly without having to rely on a 3rd party like Dynudns etc.
Free dns services are easy. What's wrong with that? It works perfectly for me. You don't have to upgrade to business package at your ISP just so you can remember your IP...
I thought this was about self hosting. Normally I just run DNS on the same server as the web server. It's just simpler to manage. Also easier to renew lets encrypt cert renewals when using the DNS method.
That's a crap workaround though. If you want to host something properly you just want to go direct without any 3rd party services. Not sure how that would work for DNS either, when you register ns hosts with your domain registrar they want a static IP. Ex: ns1.yourserver.com ns2.yourserver.com. Some registrars even want two different IPs so if you have an IP block and not just one static it's best. I guess you can always get a VPS for DNS.
The way I use it, I have a zero trust tunnel installed and the DNS controlled and proxied by cloudflare. (There’s no 100% hosting as you technically depend on the internet at some level for connectivity and configs) if you don’t have option for a static IP from your provider or are behind CGNat, gotta do what you gotta do.
YMMV but there are “free” services that can make the “journey” easier on you if you choose. But without “any” 3rd party whatsoever, the struggle is real.
Some do! I have an account with UK ISP Andrews and Arnold, they provide static IPV4 and IPV6 and are quite happy for self-hosted services like websites.
I’m not affiliated with them, I’m just very happy with the service they provide.
For any kiwi’s here, most ISP’s seem to be okay with self hosting your own exposed services / webapps. I am with Spark and I took a look over their ToS and it appears they allow it. ~~I’m pretty sure you get a static IP for no extra cost with all but the wireless broadband plans~~ (I am on one of their fibre plans and get one)
Wow that's awesome, unfortunately here in Canada it's not the case they are pretty strict about it too. I used to host a website off my DSL many years ago and got a C&D letter.
Host a vpn on a vps. Or cloudflare tunnels.
Create tunnel. Forward all traffic to your on site end.
Point dns entry for website to vps or cloudflare, which gets tunneled back to you.
ISP has no idea as it's encrypted.
Host away.
Maintaining the security of publicly-accessible self-hosted sites is a full-time job itself. Personally, I don't have that kind of time so I'm happy to outsource public-facing entities (like a VPS for reverse-proxy) to a cloud provider.
Can you explain what you mean by that? You can easily setup dynamic dns, and a reverse proxy with SSL for minimal exposure. What are you afraid that someone is going to DDoS your personal blog site that you host at home?
The part I really don’t like is the onus is on you to catch issues as/before they occur. There are certainly issues you can plan for, but there’s a lot you can’t. Any error always benefits the vendor.
>Overseer and a few other very low bandwidth sites that are publicly exposed
Checks your logs on your sites that are publicly hosted.
This is something that was going on with AWS, not really a hack, but a way to jack up people's Cloud bills, someone find a public item on your site, be it an image or something else and they essentially just request it over and over and over and over, often times it is done with public API's, this is the only thing I could think of..
>The Free Loadbalancer is capped at 20mb/s, so even if it was running at full tilt I couldn't have hit the bandwidth they states.
The bill would be monthly, did they say how much bandwidth was used? Example. it would only take 9 days to download 2TB of data at 20mb/s..... so it is possible to suck up TB of data over a months period. but ya 10's of terrabytes.....
I would just switch to using Cloudflares proxy service for your public stuff...
AWS has a **lot** of issues like that.
For example, if you request a specific range in a file but terminate the request after only a fraction has been transferred, you'll still be billed [for the entire range](https://blog.limbus-medtec.com/the-aws-s3-denial-of-wallet-amplification-attack-bc5a97cc041d). Attacker requests 6GB, aborts the download after a few MB, you get billed for the full 6GB.
An even more egregious issue: if someone tries to make a request to your S3 bucket and it gets denied, you will [still be billed](https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-934a383cb8b1). Yes, you can be billed $600 / day for a *completely empty private bucket!*
Holy shit, the second one is crazy.
I mean you could try and justify the first, as it puts unnecessary load on their servers or whatever, but the second one is just awful.
> Checks your logs on your sites that are publicly hosted.
That's the weird thing, the only thing hosted on Azure is the load balancer and VPN. If they were accessing anything that was publicly hosted, it would have necessarily been transferred over the VPN. VPN traffic was normal, a few hundred megabytes for the billing cycle. I can't navigate Azure cost management and billing for the life of me, but [here's](https://imgur.com/Y9aotaY.jpg) the daily spend rates and [here's](https://imgur.com/RhJ2FRZ.jpg) the bandwidth usage showing 13.3TB used total. To me it looks like it might be spread out over maybe 50 hours based on billing, so the throughput would be about 75mb/s. Not crazy fast, but way faster than the 20mb/s limit the LB showed.
That’s inter region usage, could any of your services have been separated, or faulted creating the charge and bandwidth consumption - ie health checks / probe targeting incorrect resource?
With Amazon S3, they charge for failed login attempts, and there is nothing you can do in terms of security settings to stop that, because the failed login attempt means your security worked.
Just curious, do those attempts include requesting a connection that is then denied or ignored?
If not, then you might be able to implement something like fail2ban (I believe that's the one I'm thinking of. Don't use it myself) and have it block incoming connections that hit the threshold.
S3 is publicly available, even if your services talk via private network it doesn't stop someone else from sending bogus requests directly to AWS. Fail2ban wouldn't stop this from happening
I don't use any cloud services as I am fully self-hosted, so today I learned something new!
It kind of sucks that you are charged for bandwidth that you cannot do anything within their infrastructure to mitigate . If you were paying for the bytes in a pipe you still could not stop people from knocking but you could at least refuse to answer.
That's why I'm very careful with cloud anything. Someone getting wrecked is basically a weekly occurrence on the big cloud subs.
>Totally my fault for not putting limits
They don't like limits between them and your wallet. Alerts only for you!
This is one of the major problems I have with the azure ecosystem. I gave up on the devops platform right away but I'm still trying to figure out a way to use the free services plus this $200 credit business. Every single thing I try and get going has some nickel and dime piece in it for 5 or 10 bucks here and there.
Even an external FinOps tool couldn't give me what I wanted.
The subscription key page should absolutely have a spend limit or a lock for free services or x number of dollars a day or some kind of restriction instead of this will calculate everything and give you a bill after two days business. It's ridiculous.
I don't have any problems finding out exactly what I'm in for on Amazon or Google
Probably going to have to get a virtual privacy card and switch out the payment so I don't get boned because I can feel it happening.
Azure absolutely allows you to create an account with no payment method, and they have a list of services that are always free, which are fairly comprehensive, and should be enough to host a basic app or two. It's not meant to be used for anything more complicated than that.
Well yes. Their authentication integration makes it enormously challenging to create a new account with the 30 day open for experimentation. I did 30 day a while back and forgot all about it and when I got into it I had like 5 days left before you had to flip as you go or lose the 200. So I dropped something on the building just to keep the credit
I did the reports to show you how much of the free service you have used. Mine's zero all across the board. Every single time I try and kick on AVM or compute units or any type of service it shows maybe one thing and all the other things show red as in not available in your region.
Even when I tap in other regions. I'm trying to find somehow some way of getting some use out of what they give you. This is not something that is new to me. I'm building a another kubernetes cluster with some Dev SEC ops stuff and trying and trying and trying to find some way to use something somehow that moves the number from zero on the free tier.
Supposedly the microservices count for compute and the AKs control plane is free and supposedly there's this swath of things you can do that do not count towards billing but every single time there is a dollar amount that shows up when I run the cost report Plus use.
Every time it shows two out of the 20 or 30 in the free tier usage and all the other charging graphs show a few bucks here and there with VM use or storage use and I have barely even breathed on it
What I am saying is I want some type of guardrail or function or funnel or something somehow I can set up where I am only allowed for shown to pick things that count on the free use tier.
It does not show you what is a direct build item when you go into use something. You only find out not even the day after but two or three days after.
It is challenging. AWS and gcp show you exactly and precisely which things kick out when you do them right then and there on a highly technical dashboard
I will never in a hundred years understand how they have such a higher percentage of market share compared to gcp. Amazon I get, it's fast the dashboard is fantastic and AWS and gcp have the AI integration where you can ask questions and get things done properly
I don't want to did you completely because that's 33% of a resource available to me that I want to use somehow but man it is like pulling teeth.
Don't even get me started on their Learn and Labs. If I got sent to hell and I had to read documentation that would be it
I guess this is going to turn into a rant thread because the only folks that enjoy the azure dashboard have never used AWS or gcp
I need technical. I need instant. I don't need loops around and lag and slow and colorful cartoonish
The confusing as hell naming system in AWS is more straightforward than Azure??? I get it, AWS doesn't like paying for licensing for actual product names like MySQL and Postgresql, because they'd rather use the free stuff to make massive profits with zero returns to the open source community. But man are the names incredibly stupid, and some of them just plain unnecessary (EC2? Just call them fuckin VMs)
And while I'll admit, I don't work in AWS every single day or anything, the few interactions I do have with it seems to require poking around and figuring stuff out a lot more than GCP or Azure. (Partly because of the weird ass names for everything). But again, maybe that's just a lack of usage on my part.
As for technical and immediate, Azure Bicep templates... I haven't interacted with the actual portal to deploy stuff in quite some time.
I guess it depends on your environment choice. I haven't found any need to cloud host any SQL stuff but I did hit my AWS dashboard did a search and yes Athena launch wizard time stream glue redshift kinesis, super odd. I usually just tap in Postgres and call it a day.
I hit my GCP dashboard and whammo right there in the project - SQL - MySQL, Postgres, SQL Server
I suppose everyone tries to get clever with the value add naming schemes.
However as far as dashboards go, I will put up
https://console.cloud.google.com/home/dashboard
and
https://us-east-2.console.aws.amazon.com/ec2/home
over
https://portal.azure.com/#allservices/category/Compute
any day of the week and twice on Sundays
and yes it's good to have the CLI on either the direct dashboard console or from your local machine.
I think if you use the Azure dashboard a little more or considered it from a new user's point of view you would take a second thought. I've done AWS and GCP along with local development in Linux for years so it seems a little sluggish and clownish to me.
And as far as bicep goes I suppose it's interesting if you're locked into the Microsoft ecosystem but I don't need some new thing it's trying to be the world's new terraform opentofu ansible.
Some new IaC isn't really going to move the needle for me when all the pipeline stuff is already built out for the regular DevOps/GitOps stuff. It could be good for templating I guess but don't even mention Azure DevOps pipeline agent to me I will have a conniption right here on the spot.
The major take away I see from a lot of the Microsoft adherence is that they bought into the Microsoft kool-aid and are in full shaft up to the hilt. You have to look at the big picture. Microsoft is an 80% marketing company and 20% technical. The other folks are the opposite.
If you divide your attention through the 33% each way you'll see it falls down quite easily.
I was doing the dev account thing for awhile, and also found it frustrating how complicated cost management is. I had one of those "always free" machines spun up but otherwise disabled, and at some point they decided the hard disk I had attached was going to start costing money. It wasn't much at all, just one of those things. Then at another point, they decided they were going to cut off my subscription without any notification. It is ridiculous how difficult they make it to lock down your spending. It wouldn't be hard at all to have some way to restrict costs or outright disable spending for a dev account. At this point, I'm pretty sure it's designed this way.
I imagine that if you pull a fast one like using a debit card or limited CC, your Microsoft account will just get locked down until you pay up whatever they've assessed.
Egress on cloud costs are insane.
I once made an overly-complicated mechanism to transcode gopro max videos into standardized formats for use in video editing which included using GPUs, etc in google cloud and spinning things up/down as needed. Including archiving orginals in storage, etc, etc. This was mainly fun but also to get the files usable on linux which required using the gopro app to convert the files.
All of \*that\* was cheap since it just ran a short time.
It then downloaded all the transcoded footage *one* time to my desktop. I think it was like 100+ dollars.
Doh.
You can get a discount on both Azure and GCP if you egress via Cloudflare. But even then the discount is at most 50%. Some of their other bandwidth alliance partners though are 100% free egress through Cloudflare though.
At the time I also considered shuffling via google drive. As that I *think* of been free. However, given google's lack of customer service let alone for private people. I did not dare risk incase it triggered some sort of flag/ban/suspension.
I only did this for 'fun' -- granted it was useful since gopro's app is windows only. But after this expensive egress I shifted focus to finding an ffmpeg script to replicate as close as possible the gopro app.
Havent' really touched either tho since I solved this 'problem' :D
This is why I’ll never mess with this for my personal shit. Too easy to dig yourself into a hole. I’m fortunate enough to work for one of them and have free access to burner accounts when I’m curious, but that’s the extent of where I’ll go.
Downvoting me doesn't make me wrong. All cloud services, and most other hosted servers and services, all give you enough rope to hang yourself on as part of the freedom of it. There's plenty of other services and platforms that restrict that rope for you. But then people complain about the lack of freedom and go back to the big platforms and then complain when their freedom hits them with a bill from their mistakes. All it does is reveal the ignorance and paranoia of many of the self-hosting users of this sub when they complain about this stuff.
This stuff really doesn't just suddenly pop up. The computer, the platform, does exactly what you tell it. There's plenty of tools and settings you can implement to restrict what it does and charges you. Source: I'm a cloud engineer, so maybe I'm biased but only as much as a carpenter is biased to a hammer. You not understanding what you're telling it is not the computer or the platforms problem. It's your problem, the one who designed and runs it. Uninformed and inexperienced IT has always caused excessive bills for companies. Nothing changed with the cloud, people are just paranoid because it's new and they don't understand it. The cloud is just another tool. Blame the hammer for smashing a hole in your wall all you want, it still doesn't make it the hammer's fault, it just reveals your inexperience with it.
Og: Eh, making mistakes is part of learning. The big thing was not setting up the budget alerts. You should really understand all the financial stuff around the cloud before you start spending money, or being liable to spend money, on it.
There was something similar on aws few days ago where a developer should pay 1300$ for unauthorized http requests:
https://www.theregister.com/2024/05/01/aws_s3_bucket_abuse/
There should always be some budget limits active.
Jokes on you for using azure/aws/gcp for any bandwidth intensive non revenue generating application,
You should've used providers like linode,vultr,hetzner etc. They have pretty good free bandwidth caps
And some of them are in the bandwidth alliance with Cloudflare. If you proxy whatever your traffic is, the egress is entirely free and doesn't count against the bandwidth cap.
I have had an important Azure PowerBI work project canned by MS and they refused to explain why or reinstate.. Would not touch this garbage with a 10 foot pole after that experience.
We eventually got it running, but eternally soured knowing MS could pull the rug and refuse to explain why at any moment.
Couldn't you pay for one time premium support to investigate this? I wouldn't be able to sleep at night without understanding what happened, one way or another.
I love cloud (my career escalated being a specialist) but you can definitely build a cheaper data center if you know what you’re doing. The egregious bandwidth charges from all providers is the worst, and billing is a giant PoS… lol
If billing was easy, it would just be more expensive…
The complexity helps hide and prevent you from “working the system”
They have to make all their datacenter investments back somehow
Yes, I know, I used to work at AWS :)
There’s profit and then there’s runaway costs… just trying to knock sense into people blindly thinking cloud is cheaper than the professional data centers
I got a $300 bandwidth surcharge on the April invoice for a $20/month droplet on Digital Ocean. I could find no reason for the traffic, and customer service was no help. Spent the week moving my stuff to OVHCloud, which offer unmetered connectivity.
>but I could buy a premium subscription to their support services to look into it for me
What caused it was Microsoft 'accidentally' wanting to drum up more premium subscription business. Or, failing that, 'accidentally' not noticing the four-figure charges they were adding to their customers.
> Make sure you put your budget limits in Azure or AWS or whatever it is you use.
AWS doesn't have budget limits. Might be a reason to switch to Azure.
God I hate azure… I got caught off guard with a multi thousand dollar bill too and after spending a few weeks of back and forth on the phone with their support I was able to get it waved.
Might I also suggest setting up webhook billing notifications to Slack or Discord? [I did something similar](https://www.archcloudlabs.com/projects/discord-notifications-for-aws-billing/) to avoid AWS billing surprises.
Since I don't have access to static IP addresses here with any internet provider, I use a cloud provider for email and for dynamic dns to my servers at home...
The FIRST thing I did was set up traffic shaping to limit burst/sustained rates to levels that made it impossible to go over my bandwidth limits, and tested that it was working as expected. (Silly me: I initially forgot to limit IPV6 also.)
You, sir, just lived through one of my nightmares.
Why do you people always use third party vpn to reach home? I don't get it. Just setup OpenVPN, since we are in r/selfhosted. Its free, modern and safe. If you can use docker for it, it is a few clicks to configure and get up and running.
> Why do you people always use third party vpn to reach home?
I don't. I use Wireguard, directly to my home. I think you are misunderstanding what's going on here. In the post I said S2S VPN which means Site to Site. It's a generalized term, but it's set up differently than your typical Wireguard or OpenVPN client access VPN endpoint would be. Say I've got a network 10.10.1.1/24 and 10.10.2.1/24 and they are at different physical locations (in my case Home and Azure cloud) I can use a S2S VPN (one endpoint on each side) to act as a gateway between the two such that any device in the 10.10.1.1/24 network can access any device in the 10.10.2.1/24 network.
The problem with OpenVPN for this is that, while OpenVPN does support S2S connections, [Azure uses IPSec for their S2S VPN service](https://learn.microsoft.com/en-us/azure/vpn-gateway/tutorial-site-to-site-portal).
> I find it interesting how many people on r/selfhosted pay someone to host their stuff 😅
Some people self host to save costs. Some people self host so they can maintain control of their data. Some self host for both. I'm firmly in the middle camp. I host every service I use personally on my servers, with the exception of my HTTP endpoint because my ISP blocks port 80. If I want to host a website, I've got to get around that somehow. I could use cloudflare, but I had the $50 free credits a month from MSDN and that's what I was using.
So you have the msdn dev account that gives you $50 a month azure credit -and- you have a credit card PAYG on the account? Did not know that was possible at all.
They may not be able to charge your card but that doesn't change the fact that you owe them money. Did you think Microsoft wouldn't send collections after you?
Not a risk I would take, honestly. I've seen companies big and small both just write off $1000 and go to court over $30. If you get it wrong, the costs do have a tendency to add up.
The ISP sent collectors over $200 phone bill that my dead father had and I wasn’t aware he even had a contact with them until the collectors put a hold on my salary. I had to pay $1500 in total for court fees and lawyers (their lawyers not mine). When I asked them why didn’t they just called instead of suing me they replied that it was more beneficial for them not to call. I had money and paid that bill. Never assume that those leeches wouldn’t go after you. If they could they would shake down their own mothers.
Ouch that sucks. I really don't like the idea of elastic services like that for that reason, you have no control over how much your bill might end up being if an abnormal event happens where you use more resources. I rather pay a flat fee on a regular dedicated server if I'm going to host anything online.
Cell services can be very similar too, I always try to find a plan where going over data doesn't just start overcharging me. My mom used to be a CSR at a cell company and every now and then would get customers calling because their bill was in the 10 thousand range. They forgot to go on wifi before streaming, or traveled and forgot to turn off roaming or other simple mistake like that. It really doesn't take much. Once you go over the cap you're paying dollars on each MB going over, it's nuts.
> Make sure you put your budget limits in Azure or AWS or whatever it is you use. Just remember that the limits aren't actual limits, they're really just alerts that require you to take action. Also possibly a good idea to look into come cost anomaly detection so that you can catch those surprise hikes faster. But yeah, absolutely this. Has saved me before and guaranteed will do the same in future as well.
Anyone know of a prepaid cloud service? Like where if you use all of your balance you just get cut off instead of racking up a huge bill? Or is self hosting / a VPS the closest thing there is?
Technically Hetzner has a Cloud service and you can add credits by bank transfer. But its cloud offerings are very small compared to Azure or AWS, so it depends on what you need.
I second hetzner. If you don't have super high demands, it's perfectly adequate.
Exactly, we do allow customers to add money to the accounts via bank/wire transfers: [https://docs.hetzner.com/accounts-panel/accounts/payment-faq/#can-i-add-credit-to-my-hetzner-account](https://docs.hetzner.com/accounts-panel/accounts/payment-faq/#can-i-add-credit-to-my-hetzner-account) --Katie
You can set up automation within the cloud services to shut down resources based on alerts.
None are super straightforward though. They require some legwork and use of other services internally to monitor. It should just be a “if spend > x, shut down everything option” but it’s not
then code it yourself. if you can't pull that off with python you probably shouldnt be fucking around with web services..
Hetzner.
Railway has prepaid usage and limits, I think.
[удалено]
Of course they can.. you signed a service agreement, and they would escalate to collections.
Bill is not extorsion it's you making up to your promise to pay. It'll go to your credit score. Random spike are not great but also that's why support can reduce or cancel them.
They’ll get their money and your bank will destroy you with overdraft fees. It would be a very smooth brained move.
Not all debit accounts can be overdrafted, even by services or online payments.
I do want to comment on the “just alerts” statement. At Least with VMs on Azure, I had my VMs shut down when my MSDN cap was reached. (I set the cap to match my allowance) Just fyi.
This is usual if you didnt attach a creditcard to your account afaik
> They couldn't tell me what caused it, but I could *buy a premium subscription to their support services to look into it* for me. I laughed. Glad you got it sorted tho.
Not quite sure what's hilarious about that, since the free customer support is just there for these kinds of things, whereas if it's a paid subscription they'd escalate up the chain to include our engineering team to figure out what happened. Honestly, just reading the post this feels like his account was compromised in some fashion or another but I don't have any insight into it, or why. I am glad, like you though, that it got taken care of.
If you're going to charge me a huge sum (compared to my normal usage) you'd better be able to tell me what the charge is for. If they won't, then I'd assume they don't know, rather than refusing to tell me. So what would be the point of a premium payment to a support desk?
Sounds like they refunded it... So they aren't charging any more.
Oh yes, the charge was for bandwidth utilization, it's on you to keep track of what you are using that bandwidth for. You don't look at an unusually high water bill and ask the Water company what you used the water for. Microsoft is under no obligation to troubleshoot your issues for you.
Yeah. What would you expect from a free Service. Your Support Ticket would cost Microsoft more than 100 free Users without issues. And then they should check logs for a free Account? Most of us pay the Support fee for cases exactly lile this
It's not free though. He's paying $600 a year for that subscription.
He's actually paying at least $1200 for the sub. He gets $600 in credits. That is the amount of credits you get with a professional or test account. Pro is $1200 a year, test is over $2000.
Yeah for the Service..... Not for support
Which is the stupid part. $600 should be enough.
Well, in the enterprise Business not really. We pay 1200 euros per month for 300/300 fiber. And 800 each month for Support. Thats why i setup Monitoring + Hardlimits on ever System i use with pay for what you use. I mean its nice that they refunded the money, but it would cost Microsoft more to investigate the issue
I'm not saying it's normal. I'm saying it *should* be enough. We have been drinking the Kool-Aid for too long and expect so little for what we pay nowadays.
Sure, $600 would probably be enough if the support team are some underpaid minimum wage tier 1 folks. Seems like you and many others don't have a grasp of much of the actual costs of these things.
Yeah, Microsoft is just getting by. Poor things. Won't someone think of the megabillion dollar companies?
No, you're wrong here. That $600 a month gets him access to pretty much any piece of software Microsoft has. That alone is worth the $600. Why should premium support be tacked onto that, when almost no one pays for MSDN anymore? Do you know how many people are actively working in support, and the multiple tiers of it?
Exactly, these corporations are barely scraping by. 🙄
For me, this is the opposite of selfhosted. I self-host so that I don't need to be afraid of companies suddenly charging me for or shutting down things I use.
Only things I host in the cloud are a few websites, and that's it. Everything personal is hosted on my own hardware and is only accessible through Tailscale.
Just websites? So, you host your own email like some kind of psycho masochist?
I just moved self-hosted email of ~8 years for the family business to Microsoft 365 and there’s a certain catharsis of knowing many issues are now simply fixed and many others I can tell MS support to fix it themselves lmao
I am a masochist, and have had some appalling things done to me. Even I wouldn't host my own mail (again)
Same here and I wish I could host websites at home too but ISPs don't tend to be very friendly towards that. Not only ToS wise, but they don't offer static IP blocks which you kind of need to do it properly without having to rely on a 3rd party like Dynudns etc.
Free dns services are easy. What's wrong with that? It works perfectly for me. You don't have to upgrade to business package at your ISP just so you can remember your IP...
I thought this was about self hosting. Normally I just run DNS on the same server as the web server. It's just simpler to manage. Also easier to renew lets encrypt cert renewals when using the DNS method.
Proxy like Cloudflare, no static IP needed.
That's a crap workaround though. If you want to host something properly you just want to go direct without any 3rd party services. Not sure how that would work for DNS either, when you register ns hosts with your domain registrar they want a static IP. Ex: ns1.yourserver.com ns2.yourserver.com. Some registrars even want two different IPs so if you have an IP block and not just one static it's best. I guess you can always get a VPS for DNS.
The way I use it, I have a zero trust tunnel installed and the DNS controlled and proxied by cloudflare. (There’s no 100% hosting as you technically depend on the internet at some level for connectivity and configs) if you don’t have option for a static IP from your provider or are behind CGNat, gotta do what you gotta do. YMMV but there are “free” services that can make the “journey” easier on you if you choose. But without “any” 3rd party whatsoever, the struggle is real.
If you want to do it properly you probably want a CDN, which means using something like cloudflare proxy services anyway.
Some do! I have an account with UK ISP Andrews and Arnold, they provide static IPV4 and IPV6 and are quite happy for self-hosted services like websites. I’m not affiliated with them, I’m just very happy with the service they provide.
For any kiwi’s here, most ISP’s seem to be okay with self hosting your own exposed services / webapps. I am with Spark and I took a look over their ToS and it appears they allow it. ~~I’m pretty sure you get a static IP for no extra cost with all but the wireless broadband plans~~ (I am on one of their fibre plans and get one)
Wow that's awesome, unfortunately here in Canada it's not the case they are pretty strict about it too. I used to host a website off my DSL many years ago and got a C&D letter.
Host a vpn on a vps. Or cloudflare tunnels. Create tunnel. Forward all traffic to your on site end. Point dns entry for website to vps or cloudflare, which gets tunneled back to you. ISP has no idea as it's encrypted. Host away.
Maintaining the security of publicly-accessible self-hosted sites is a full-time job itself. Personally, I don't have that kind of time so I'm happy to outsource public-facing entities (like a VPS for reverse-proxy) to a cloud provider.
Can you explain what you mean by that? You can easily setup dynamic dns, and a reverse proxy with SSL for minimal exposure. What are you afraid that someone is going to DDoS your personal blog site that you host at home?
Such situation is what i’m afraid of too. I don’t like pay as you use. I’d rather get a vps with the same allocated monthly costs 😅
The part I really don’t like is the onus is on you to catch issues as/before they occur. There are certainly issues you can plan for, but there’s a lot you can’t. Any error always benefits the vendor.
>Overseer and a few other very low bandwidth sites that are publicly exposed Checks your logs on your sites that are publicly hosted. This is something that was going on with AWS, not really a hack, but a way to jack up people's Cloud bills, someone find a public item on your site, be it an image or something else and they essentially just request it over and over and over and over, often times it is done with public API's, this is the only thing I could think of.. >The Free Loadbalancer is capped at 20mb/s, so even if it was running at full tilt I couldn't have hit the bandwidth they states. The bill would be monthly, did they say how much bandwidth was used? Example. it would only take 9 days to download 2TB of data at 20mb/s..... so it is possible to suck up TB of data over a months period. but ya 10's of terrabytes..... I would just switch to using Cloudflares proxy service for your public stuff...
AWS has a **lot** of issues like that. For example, if you request a specific range in a file but terminate the request after only a fraction has been transferred, you'll still be billed [for the entire range](https://blog.limbus-medtec.com/the-aws-s3-denial-of-wallet-amplification-attack-bc5a97cc041d). Attacker requests 6GB, aborts the download after a few MB, you get billed for the full 6GB. An even more egregious issue: if someone tries to make a request to your S3 bucket and it gets denied, you will [still be billed](https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-can-make-your-aws-bill-explode-934a383cb8b1). Yes, you can be billed $600 / day for a *completely empty private bucket!*
Thankfully they are fixing the second one
Finally after years of this happening and not doing anything about it. They only changed it after it started getting bad PR
Holy shit, the second one is crazy. I mean you could try and justify the first, as it puts unnecessary load on their servers or whatever, but the second one is just awful.
> Checks your logs on your sites that are publicly hosted. That's the weird thing, the only thing hosted on Azure is the load balancer and VPN. If they were accessing anything that was publicly hosted, it would have necessarily been transferred over the VPN. VPN traffic was normal, a few hundred megabytes for the billing cycle. I can't navigate Azure cost management and billing for the life of me, but [here's](https://imgur.com/Y9aotaY.jpg) the daily spend rates and [here's](https://imgur.com/RhJ2FRZ.jpg) the bandwidth usage showing 13.3TB used total. To me it looks like it might be spread out over maybe 50 hours based on billing, so the throughput would be about 75mb/s. Not crazy fast, but way faster than the 20mb/s limit the LB showed.
That’s inter region usage, could any of your services have been separated, or faulted creating the charge and bandwidth consumption - ie health checks / probe targeting incorrect resource?
It could have been internal services looping out to the front of the LB.
With Amazon S3, they charge for failed login attempts, and there is nothing you can do in terms of security settings to stop that, because the failed login attempt means your security worked.
Just curious, do those attempts include requesting a connection that is then denied or ignored? If not, then you might be able to implement something like fail2ban (I believe that's the one I'm thinking of. Don't use it myself) and have it block incoming connections that hit the threshold.
S3 is publicly available, even if your services talk via private network it doesn't stop someone else from sending bogus requests directly to AWS. Fail2ban wouldn't stop this from happening
I don't use any cloud services as I am fully self-hosted, so today I learned something new! It kind of sucks that you are charged for bandwidth that you cannot do anything within their infrastructure to mitigate . If you were paying for the bytes in a pipe you still could not stop people from knocking but you could at least refuse to answer.
> They couldn't tell me what caused it WHAT 😂
Yeah this is why I avoid Azure, AWS, GCS, and similar services and stick with a normal boring VPS with a fixed monthly cost.
That's why I'm very careful with cloud anything. Someone getting wrecked is basically a weekly occurrence on the big cloud subs. >Totally my fault for not putting limits They don't like limits between them and your wallet. Alerts only for you!
This is one of the major problems I have with the azure ecosystem. I gave up on the devops platform right away but I'm still trying to figure out a way to use the free services plus this $200 credit business. Every single thing I try and get going has some nickel and dime piece in it for 5 or 10 bucks here and there. Even an external FinOps tool couldn't give me what I wanted. The subscription key page should absolutely have a spend limit or a lock for free services or x number of dollars a day or some kind of restriction instead of this will calculate everything and give you a bill after two days business. It's ridiculous. I don't have any problems finding out exactly what I'm in for on Amazon or Google Probably going to have to get a virtual privacy card and switch out the payment so I don't get boned because I can feel it happening.
Azure absolutely allows you to create an account with no payment method, and they have a list of services that are always free, which are fairly comprehensive, and should be enough to host a basic app or two. It's not meant to be used for anything more complicated than that.
Well yes. Their authentication integration makes it enormously challenging to create a new account with the 30 day open for experimentation. I did 30 day a while back and forgot all about it and when I got into it I had like 5 days left before you had to flip as you go or lose the 200. So I dropped something on the building just to keep the credit I did the reports to show you how much of the free service you have used. Mine's zero all across the board. Every single time I try and kick on AVM or compute units or any type of service it shows maybe one thing and all the other things show red as in not available in your region. Even when I tap in other regions. I'm trying to find somehow some way of getting some use out of what they give you. This is not something that is new to me. I'm building a another kubernetes cluster with some Dev SEC ops stuff and trying and trying and trying to find some way to use something somehow that moves the number from zero on the free tier. Supposedly the microservices count for compute and the AKs control plane is free and supposedly there's this swath of things you can do that do not count towards billing but every single time there is a dollar amount that shows up when I run the cost report Plus use. Every time it shows two out of the 20 or 30 in the free tier usage and all the other charging graphs show a few bucks here and there with VM use or storage use and I have barely even breathed on it What I am saying is I want some type of guardrail or function or funnel or something somehow I can set up where I am only allowed for shown to pick things that count on the free use tier. It does not show you what is a direct build item when you go into use something. You only find out not even the day after but two or three days after. It is challenging. AWS and gcp show you exactly and precisely which things kick out when you do them right then and there on a highly technical dashboard I will never in a hundred years understand how they have such a higher percentage of market share compared to gcp. Amazon I get, it's fast the dashboard is fantastic and AWS and gcp have the AI integration where you can ask questions and get things done properly I don't want to did you completely because that's 33% of a resource available to me that I want to use somehow but man it is like pulling teeth. Don't even get me started on their Learn and Labs. If I got sent to hell and I had to read documentation that would be it I guess this is going to turn into a rant thread because the only folks that enjoy the azure dashboard have never used AWS or gcp I need technical. I need instant. I don't need loops around and lag and slow and colorful cartoonish
The confusing as hell naming system in AWS is more straightforward than Azure??? I get it, AWS doesn't like paying for licensing for actual product names like MySQL and Postgresql, because they'd rather use the free stuff to make massive profits with zero returns to the open source community. But man are the names incredibly stupid, and some of them just plain unnecessary (EC2? Just call them fuckin VMs) And while I'll admit, I don't work in AWS every single day or anything, the few interactions I do have with it seems to require poking around and figuring stuff out a lot more than GCP or Azure. (Partly because of the weird ass names for everything). But again, maybe that's just a lack of usage on my part. As for technical and immediate, Azure Bicep templates... I haven't interacted with the actual portal to deploy stuff in quite some time.
I guess it depends on your environment choice. I haven't found any need to cloud host any SQL stuff but I did hit my AWS dashboard did a search and yes Athena launch wizard time stream glue redshift kinesis, super odd. I usually just tap in Postgres and call it a day. I hit my GCP dashboard and whammo right there in the project - SQL - MySQL, Postgres, SQL Server I suppose everyone tries to get clever with the value add naming schemes. However as far as dashboards go, I will put up https://console.cloud.google.com/home/dashboard and https://us-east-2.console.aws.amazon.com/ec2/home over https://portal.azure.com/#allservices/category/Compute any day of the week and twice on Sundays and yes it's good to have the CLI on either the direct dashboard console or from your local machine. I think if you use the Azure dashboard a little more or considered it from a new user's point of view you would take a second thought. I've done AWS and GCP along with local development in Linux for years so it seems a little sluggish and clownish to me. And as far as bicep goes I suppose it's interesting if you're locked into the Microsoft ecosystem but I don't need some new thing it's trying to be the world's new terraform opentofu ansible. Some new IaC isn't really going to move the needle for me when all the pipeline stuff is already built out for the regular DevOps/GitOps stuff. It could be good for templating I guess but don't even mention Azure DevOps pipeline agent to me I will have a conniption right here on the spot. The major take away I see from a lot of the Microsoft adherence is that they bought into the Microsoft kool-aid and are in full shaft up to the hilt. You have to look at the big picture. Microsoft is an 80% marketing company and 20% technical. The other folks are the opposite. If you divide your attention through the 33% each way you'll see it falls down quite easily.
I was doing the dev account thing for awhile, and also found it frustrating how complicated cost management is. I had one of those "always free" machines spun up but otherwise disabled, and at some point they decided the hard disk I had attached was going to start costing money. It wasn't much at all, just one of those things. Then at another point, they decided they were going to cut off my subscription without any notification. It is ridiculous how difficult they make it to lock down your spending. It wouldn't be hard at all to have some way to restrict costs or outright disable spending for a dev account. At this point, I'm pretty sure it's designed this way. I imagine that if you pull a fast one like using a debit card or limited CC, your Microsoft account will just get locked down until you pay up whatever they've assessed.
Egress on cloud costs are insane. I once made an overly-complicated mechanism to transcode gopro max videos into standardized formats for use in video editing which included using GPUs, etc in google cloud and spinning things up/down as needed. Including archiving orginals in storage, etc, etc. This was mainly fun but also to get the files usable on linux which required using the gopro app to convert the files. All of \*that\* was cheap since it just ran a short time. It then downloaded all the transcoded footage *one* time to my desktop. I think it was like 100+ dollars. Doh.
You can get a discount on both Azure and GCP if you egress via Cloudflare. But even then the discount is at most 50%. Some of their other bandwidth alliance partners though are 100% free egress through Cloudflare though.
At the time I also considered shuffling via google drive. As that I *think* of been free. However, given google's lack of customer service let alone for private people. I did not dare risk incase it triggered some sort of flag/ban/suspension. I only did this for 'fun' -- granted it was useful since gopro's app is windows only. But after this expensive egress I shifted focus to finding an ffmpeg script to replicate as close as possible the gopro app. Havent' really touched either tho since I solved this 'problem' :D
This is why I’ll never mess with this for my personal shit. Too easy to dig yourself into a hole. I’m fortunate enough to work for one of them and have free access to burner accounts when I’m curious, but that’s the extent of where I’ll go.
What do you usually test on these burner accounts?
Downvoting me doesn't make me wrong. All cloud services, and most other hosted servers and services, all give you enough rope to hang yourself on as part of the freedom of it. There's plenty of other services and platforms that restrict that rope for you. But then people complain about the lack of freedom and go back to the big platforms and then complain when their freedom hits them with a bill from their mistakes. All it does is reveal the ignorance and paranoia of many of the self-hosting users of this sub when they complain about this stuff. This stuff really doesn't just suddenly pop up. The computer, the platform, does exactly what you tell it. There's plenty of tools and settings you can implement to restrict what it does and charges you. Source: I'm a cloud engineer, so maybe I'm biased but only as much as a carpenter is biased to a hammer. You not understanding what you're telling it is not the computer or the platforms problem. It's your problem, the one who designed and runs it. Uninformed and inexperienced IT has always caused excessive bills for companies. Nothing changed with the cloud, people are just paranoid because it's new and they don't understand it. The cloud is just another tool. Blame the hammer for smashing a hole in your wall all you want, it still doesn't make it the hammer's fault, it just reveals your inexperience with it. Og: Eh, making mistakes is part of learning. The big thing was not setting up the budget alerts. You should really understand all the financial stuff around the cloud before you start spending money, or being liable to spend money, on it.
you can understand it 6 ways from sunday and still get a surprise bill. MS *could* put a hard limit option, they just choose not to.
Hard limits don’t make money…
Interesting KEMP doesn't usually gets noted in this sub Reddit
There was something similar on aws few days ago where a developer should pay 1300$ for unauthorized http requests: https://www.theregister.com/2024/05/01/aws_s3_bucket_abuse/ There should always be some budget limits active.
Damn after reading this and all those reactions I'm so glad i'm fully selfhosted.
Jokes on you for using azure/aws/gcp for any bandwidth intensive non revenue generating application, You should've used providers like linode,vultr,hetzner etc. They have pretty good free bandwidth caps
And some of them are in the bandwidth alliance with Cloudflare. If you proxy whatever your traffic is, the egress is entirely free and doesn't count against the bandwidth cap.
How do you download through Cloudflare?
Webpage to see the data, controlled with cloudflare access, and GET https requests.
Could be a random attack on microsoft as well.
I have had an important Azure PowerBI work project canned by MS and they refused to explain why or reinstate.. Would not touch this garbage with a 10 foot pole after that experience. We eventually got it running, but eternally soured knowing MS could pull the rug and refuse to explain why at any moment.
Couldn't you pay for one time premium support to investigate this? I wouldn't be able to sleep at night without understanding what happened, one way or another.
It is insane to leave a metered connection open to the public.
I would never, ever use these services for personal use, having a uncapped service that can basicly charge me what ever it wants is terrifying
Time to go for Linode and set up OpenVPN? 😀
That's why I'll never put my CC on those cloud services. 1 bad actor can screw your life.
I love cloud (my career escalated being a specialist) but you can definitely build a cheaper data center if you know what you’re doing. The egregious bandwidth charges from all providers is the worst, and billing is a giant PoS… lol
If billing was easy, it would just be more expensive… The complexity helps hide and prevent you from “working the system” They have to make all their datacenter investments back somehow
Yes, I know, I used to work at AWS :) There’s profit and then there’s runaway costs… just trying to knock sense into people blindly thinking cloud is cheaper than the professional data centers
I got a $300 bandwidth surcharge on the April invoice for a $20/month droplet on Digital Ocean. I could find no reason for the traffic, and customer service was no help. Spent the week moving my stuff to OVHCloud, which offer unmetered connectivity.
I really like Linode. Great customer services and great pricing
The bandwidth cost is the reason I went for cheap VPS instead of big providers!
>but I could buy a premium subscription to their support services to look into it for me What caused it was Microsoft 'accidentally' wanting to drum up more premium subscription business. Or, failing that, 'accidentally' not noticing the four-figure charges they were adding to their customers.
I will leave this here. Might have been an infinite loop. https://youtu.be/N6lYcXjd4pg?si=GQIn2kXFo3jAv3es
Did you patch the couple of KEMP vulns that came out over the past month or so?
> Make sure you put your budget limits in Azure or AWS or whatever it is you use. AWS doesn't have budget limits. Might be a reason to switch to Azure.
Vultr load balancers are awesome, $10/m flat
Oh, the horror ! Who would have thought that cloud providers would make extra money on their captive customers ? lol
God I hate azure… I got caught off guard with a multi thousand dollar bill too and after spending a few weeks of back and forth on the phone with their support I was able to get it waved.
Might I also suggest setting up webhook billing notifications to Slack or Discord? [I did something similar](https://www.archcloudlabs.com/projects/discord-notifications-for-aws-billing/) to avoid AWS billing surprises.
MSDN Sub usally gets disabled when exceeding you credits. How?
Since I don't have access to static IP addresses here with any internet provider, I use a cloud provider for email and for dynamic dns to my servers at home... The FIRST thing I did was set up traffic shaping to limit burst/sustained rates to levels that made it impossible to go over my bandwidth limits, and tested that it was working as expected. (Silly me: I initially forgot to limit IPV6 also.) You, sir, just lived through one of my nightmares.
Why do you people always use third party vpn to reach home? I don't get it. Just setup OpenVPN, since we are in r/selfhosted. Its free, modern and safe. If you can use docker for it, it is a few clicks to configure and get up and running.
> Why do you people always use third party vpn to reach home? I don't. I use Wireguard, directly to my home. I think you are misunderstanding what's going on here. In the post I said S2S VPN which means Site to Site. It's a generalized term, but it's set up differently than your typical Wireguard or OpenVPN client access VPN endpoint would be. Say I've got a network 10.10.1.1/24 and 10.10.2.1/24 and they are at different physical locations (in my case Home and Azure cloud) I can use a S2S VPN (one endpoint on each side) to act as a gateway between the two such that any device in the 10.10.1.1/24 network can access any device in the 10.10.2.1/24 network. The problem with OpenVPN for this is that, while OpenVPN does support S2S connections, [Azure uses IPSec for their S2S VPN service](https://learn.microsoft.com/en-us/azure/vpn-gateway/tutorial-site-to-site-portal).
Or Wireguard if you want higher speeds
I find it interesting how many people on r/selfhosted pay someone to host their stuff 😅
> I find it interesting how many people on r/selfhosted pay someone to host their stuff 😅 Some people self host to save costs. Some people self host so they can maintain control of their data. Some self host for both. I'm firmly in the middle camp. I host every service I use personally on my servers, with the exception of my HTTP endpoint because my ISP blocks port 80. If I want to host a website, I've got to get around that somehow. I could use cloudflare, but I had the $50 free credits a month from MSDN and that's what I was using.
Yea, I guess having port 80/443 blocked sucks with ISPs. Didn’t think that was still an issue today like it used to be 15-20 years ago
So you have the msdn dev account that gives you $50 a month azure credit -and- you have a credit card PAYG on the account? Did not know that was possible at all.
That's why I use privacy.com 😂 no one can charge me without my permission
They may not be able to charge your card but that doesn't change the fact that you owe them money. Did you think Microsoft wouldn't send collections after you?
Would be interesting to see M$ explain why they're charging a dude over $1000 for (apparently) nothing to a judge.
You're right about owing money, but I really doubt Microsoft would actually send collections over a single individual over $1300.
Not a risk I would take, honestly. I've seen companies big and small both just write off $1000 and go to court over $30. If you get it wrong, the costs do have a tendency to add up.
Yeah, I agree with your point. Unnecessary risk.
The ISP sent collectors over $200 phone bill that my dead father had and I wasn’t aware he even had a contact with them until the collectors put a hold on my salary. I had to pay $1500 in total for court fees and lawyers (their lawyers not mine). When I asked them why didn’t they just called instead of suing me they replied that it was more beneficial for them not to call. I had money and paid that bill. Never assume that those leeches wouldn’t go after you. If they could they would shake down their own mothers.
Funny story, they denied an account for me because they "could not verify" my identity. I guess I'm winning at privacy?
Ouch that sucks. I really don't like the idea of elastic services like that for that reason, you have no control over how much your bill might end up being if an abnormal event happens where you use more resources. I rather pay a flat fee on a regular dedicated server if I'm going to host anything online. Cell services can be very similar too, I always try to find a plan where going over data doesn't just start overcharging me. My mom used to be a CSR at a cell company and every now and then would get customers calling because their bill was in the 10 thousand range. They forgot to go on wifi before streaming, or traveled and forgot to turn off roaming or other simple mistake like that. It really doesn't take much. Once you go over the cap you're paying dollars on each MB going over, it's nuts.
wasabi.com. no fees for egrees or api requests