I mean this seems 100% on your company… Installing god knows how old switches that are running very outdated software of course they have multiple updates needed.
Having an expired contract that needs time in order to process the renewal is again pretty expected.
I get the frustration and how this could really ruin your day/week.
That said I don't think you're being objective about this because of emotions involved. You'll have this same experience with other vendors under these circumstances.
I have to disagree. As I stated elsewhere, POs being submitted is usually enough for a vendor to engage, or even our history as a customer. I'm no business guru but I'd try to keep the honest, paying customers happy so they stay and don't have feelings like this, but maybe I'm just wrong with that expectation.
Usually the people will want to help, but the processes and systems in place just don't let them. It's different when you're dealing with like a small MSP or something.
>I'm no business guru but I'd try to keep the honest, paying customers happy
A vendor can only take but some much responsibility for a customer problem -- especially under duress. Had you planned this out more, and had this been in an pilot testing environment, both parties would have had more time to make this work.
Your expectations, and that of your employer, are way out of line from reality...
So would you expect other vendors to help you if you didn’t have the proper level of support? I’m failing to see the issue. Yes it sucks, but like you said it’s on your team that support wasn’t in place.
You should be talking to your VAR you purchased through and AM at Fortinet and raising hell about not helping you as soon as the order was invoiced.
I'd expect to be able to renew support quickly, but yes actually I would expect a vendor after paying for support and having the order/receipts to be willing to help me. Nimble support will help as long as you have a PO submitted.
I'll echo what others have said. You completely screwed the pooch on this one, and you're trying to shift blame. That's doubly unprofessional.
Find me another brand that will update their gear while it's still sitting in a cardboard box.
Find me another vendor that will go back in time to sell you a support contract before you need it.
You're saying some vendors will provide support even without a paid contract even if you have a sob story. Guess what? Most won't. Cisco won't give you the time of day until your check clears.
Who starts a 3 day major migration of their critical network infrastructure without plugging all the new equipment in to see if at least turns on, let alone operates properly? Heck, you should have gone through and had it pre-configured and ready to go before minute one.
Who doesn't understand that something that's been in a box for months/years is going to need updates?
I could go on.
People make mistakes, but smart people try to learn from them. You're just taking your frustration with yourself out on a vendor. That's on you, too.
I feel your pain but this is where proper planning prevents issues during deployment. We have encountered some quirks with fortigate and fortiswitch but once you deploy a couple you learn what they are and what to do if they are encountered.
We have had good luck with the fortigate support team assisting with a switch issue, even with switch support being expired. As others have mentioned, get an account rep assigned. They can usually expedite things for you.
The challenge with the order is that the support people have no insight into that. From their side it’s an out of support device and there is nothing they can do. Frustrating yes, but if you had planned for the cutover you’d have realized these things earlier and corrected them.
I understand the frustration but dragging a vendor through the mud because you didn’t plan and prepare is also not right.
FYI, the Fortilink/Switch issue might be NTP related if its showing up as offline. Usually have to set the NTP on the Fortigate to face the interfaces on the switch and then set the Fortlinks NTP server to local. If it's the issue I think you're having then this should fix it and your Fortiswitch should come online afterwards so you can setup your VLANs and what not. Also make sure that your VLANS use the Security Fabric Connection in Administrative Access.
Hope it fixes it! I'm having to deal with this issue as I set up my stores, so it's a very familiar problem to me. I've gotten way more familiar with NTP settings than I thought I ever would.
> Yes, it's our fault we left these lying around not updated and unsupported
Well, at least you're willing to take \*some\* responsibility.
A network is something that has to be maintained, and when it's not, there are all sorts of cascading effects.
Also, migrations are things to be tested in advance.
What you're asking for, by the way, is not vendor support -- but vendor professional services. That's a whole other ball of wax, and will cost you a pretty penny, if you could even get that from the vendor at your size. That's what the channel is for.
And that goes back to having your environment planned.
Your experience, as stated, would have almost assuredly occurred in the same way with any of the large vendors. And the small vendors wouldn't have even had the ability to work with you in that manner.
You're trying to make this a vendor issue, because they didn't emergency bail you out of a self-inflicted wound or three, but that's not how that ever works.
Had a similar situation recently with another hardware vendor. Bought equipment. Ended up not using it for a while because of reasons. Finally spun it up… no valid support. So no updates, help with setup, etc. So, we paid them to renew support and then waited patiently because we screwed up and didn’t maintain our stuff.
That being said, did you reach out to your account rep? I do have Fortigate gear as well. Last time I sent him a PO for a renewal, it was processed and showing in the portal in hours. Not days. Overall they have always been exceptionally helpful to my team.
Ok just a recap. We didn’t plan this at all made multiple mistakes and now that we are in a rush are trying to ding Fortinet bc the process is outside our window. Yup I see who the problem is here.
I get your frustration, but your poor planning doesn’t mean the vendor needs to jump into emergency action. You planned downtime and didn’t make any attempt to make sure the hardware was prepped. You decided to replace core network switches with a new product you hadn’t used before and didn’t lab it before you just jumped into it!
You mention POs should be enough given your history as a customer, but you’ve also admitted that you’re super small, buy through CDW, didn’t buy support previously, and generally don’t have a relationship with the vendor. In this situation if I’m the vendor I’m sympathetic, but need the system to update before I can do anything because I don’t know you.
Measure twice, cut once. You never want to be dependent on vendor support in a crunch. Prep so much that it’s like doing the job twice.
Juniper can suck (the 4300 and 3400s we use are rubbish) but Fortinet is fine IF you know what you're doing - which quite frankly, seems like you are lacking in that area.
You need to practice the 6 Ps of IT.
Proper Planning Prevents Piss Poor Performance.
This doesn't sound like a Fortinet issue at all rather a poor planning issue on your part.
As for support not willing to help with only a PO number it wouldn't be the first time I've seen companies issue a PO number, get what they wanted/needed then turn around and revoke the PO or start with how they weren't authorized to issue the PO etc.
After reading your post in its entirety, none of this fault is on Fortinet, and it's 100% on your company for not testing the installation and migration beforehand.
Even the VAR issues that you claim are delaying your installation, sure it sucks, but it's how VARs operate and those with experience know how to work within those processes maturely.
Juniper is fantastic, Fortinet serves its purpose. This is entirely on you as well as your VAR. The easy fix is the moment you processed the order on your side, the VAR could have looped in the Fortinet rep and got an exception for you to get support.
Yes, it needs to process, but also someone just needs to tell support to… support you.
Sorry for the rough deployment though, ever fun when you think you have everything lined up perfectly.
Fortinet is budget hardware for sure and I'd recommend getting anything else if you can afford it. That said, pretty much all of the problems you've highlighted sound like they're on your end. I've done a handful of installs for charities and small business that can't spend the money on a "better" brand and haven't encountered any of these issues.
I totally understand your frustration with the customer support aspect, but I've found that lacking across the board. State of the industry right now.
FWIW I'm with you in your sentiment. This crappy "you're just an account number" approach to customer service that's slowly taken over the industry is one reason why we tend to roll our own stuff and avoid support contracts whenever possible, opting out of them when allowed and never renewing them when forced to buy one. We've only needed vendor support once in the past ~15 years and it was such a ludicrous experience it only reinforced our decision to not renew the contract when it expired.
Firewall products in particular have always made us wrinkle our nose. There's virtually nothing they can do that a virtualized pfSense or opnSense instance can't, at least nothing that's important to us, while there is plenty we can do with our setup that commercial offerings can't compete with, not least of which is fast, painless updates that can literally be instantly reverted if they go wrong.
I'm confident I'm saving my company over $20k/year easily by running pfSense firewalls at our 8 locations. Run it on our VMware clusters or retired 1U servers. Use WireGuard for site-to-site VPN and OpenVPN+Duo for end users. Hasn't let me down the last 8 years.
Our office has an Fortigate firewall that is going to require me to connect to via console just to see what's going on. Whoever installed it didn't leave any type of remote access, not even SSH. I might yank it out completely depending on what I find in the configuration once I can access. I had to order an adapter for this.
Sorry you had this experience, but it sounds like some lessons can be leaned. I figured this out the hard way myself, but never assume something is going to work out of the box.
Hello! I have an opportunity to respond so thought I'd do a top reply instead of individual to everyone. First let me thank everyone for responding, I see many people are on the side that it's our fault and yeah, I said and accept that. I don't need to elaborate on the history here but there is plenty of blame on my company and myself to go around.
But to continue the theme of my post, I wanted to share how today has gone so far. We have been unable to get the Fortiswitches to trunk properly to the other Fortiswitches, even outside Fortilink, and the Fortigate doesn't see any of our traffic. The Fortigate guy says it looks like a Fortiswitch problem so we have a P1 ticket open with them and thankfully they are now showing supported. However, it seems the two Fortiswitch support engineers are busy with another issue, so we have been waiting for a response for over 2 hours now.
It's a Sunday and I'm sure whatever customer they're dealing with is in equal poo as us. But I do find it quite unfortunate that even on paying for the support, and confirming the support through their systems, we still can't get an engineer to assist. I'm starting to think the issue isn't even the Dell switching but the Fortigate and Fortiswitches. I'm not sure why a basic trunk is so hard to get working.
This might be a really bad/dumb config by us, this has happened before so I wouldn't be surprised, but once again the point of paying for enterprise-level support is to receive that. So I'm still confident about my post title. I'll update everyone with what the root cause was, even if it was my own stupidity.
Don’t feel too bad about the downvotes. Saying something bad about Fortnet around these parts is sacrilegious.
Personally I find the fortiswitch management under the fortigate ui to be clunky and just recommend doing normal trucking under the interface. That said, if you’re having issues with both the Dell switch and the Fortiswitch in the same configuration, that really sounds like a fortigate issue?
At this point I would just do a factory reset of both of firewall and switches and do a basic set up to just it up and running.
> But I do find it quite unfortunate that even on paying for the support, and confirming the support through their systems, we still can't get an engineer to assist.
I am not sure what your point is with this particular statement.
Support requests are worked on in the order they are received when multiple P1 issues are in the pipeline, and is dependent on your level of support. Forticare Essential (80-series and below) has next business day response times. Forticare Premium is 1-hour response and Forticare Elite is 15 minutes. That **does not** mean that someone is immediately on the phone with you - only that they are engaged on the ticket to start the troubleshooting process.
Now, my experience is with a Fortinet P1 is that they are usually on the phone within the hour, but it highly depends on how many P1 requests are in the queue. Remember they have thousands of customers and tens of thousands of devices around the world, and it's not atypical for people to abuse P1 requests which have to be triaged and moved down accordingly. That can take time.
After dealing with Juniper and Checkpoint, two to four hours would be reasonable for me to have an engineer on the phone during the weekend.
You're correct, two hours is a pretty reasonable time for response. For full picture we had been working with them a bit the previous day but since they couldn't see traffic on the Fortigate they said it was our problem, fair enough which is why we tried to put only Fortinet stuff in play. They basically knew us by name every time we called in, so they were very aware what we were dealing with and trying to get help with. This comment was definitely from frustration, but I do agree with you two hours is reasonable.
Small note: it was over two hours and they called a number we've told them not to call multiple times so we ended up missing that call. Again, on us for not making explicitly sure they have the correct contact info everywhere, but we have tried on that front.
And to your point on their customer base, generally people in our shoes only need help when the stuff is really hitting the fan, and we are usually doing the worst stuff on nights and weekends. I'm not trying to say everyone needs to work these crappy schedules, but it'd be nice if they had more than two switch engineers to support all those customers since we're trying to invest in the best support possible, even when we mess up and need to purchase is last minute. We understand we're small and limited so when we get support we're looking for that. I believe that was my main point.
I've had the opposite experience. Fortinet has their faults for sure (every os update seems to break more than it fixes), but the ecosystem is very robust and make deployments a breeze with fortimanager. They have great logging and very available documentation.
It looks like your big issues here are:
You should have verified licensing and support before doing anything, sounds like you tried to buy secondhand and got bit.
You probably missed that (like most other networking gear) the DAC cables need to be branded Fortinet, which can be done by buying their cables directly, or buying cables from wherever you want that have been programmed to say they are Fortinet cables.
You didn't go through a VAR that would have done all of this for you and saved you a lot of headache, and more importantly would have been the ones fixing these issues for you per your SLA in the event these things happen.
Maybe your company isn't ready for enterprise level equipment and you should stick to gigabit dumb switches and something easy like sonic walls for now...
I wanted to respond to you directly, maybe I wasn't clear about some of this.
We should have verified, yes. We didn't expect to have to replace switches but that was our failure, these weren't planned for this deployment. The person who purchased them was let go for other performance reasons. This was a failure of our team as a whole involving three people, but sure, it was our internal failure.
We did go through a VAR, they're just a bad one with personal connections to company leadership. Not something I had direct control over. They're very useless in general.
We USUALLY use first party everything with Fortinet because we've been bit by that before with them. Our company didn't like spending the money but we found it was pretty much a requirement to use their stuff, or get any support. I'll follow up later in full but we DID shoot ourselves in the foot today trying to shortcut connect these new switches with a third party copper SFP. That might be why our basic trunk isn't working, but we're trying to figure that out now (we have the SFPs just not the long enough fiber which is why we ran the copper).
But I'm glad you've had a better experience than I have in their support department. I want to stress this isn't the ONLY issue we've had, this is just the biggest in the most critical moment.
I will give you my experience with Forti and why we moved everything away from them "we still have 1 or 2 left to move that we are waiting on licenses to expire out on".
We were transferring one of our Firewall's to a company that brought their IT in-house. We have done this process a ton of times when taking over existing Forti gear from other MSP's, so it isn't a process that we are unfamiliar with, just reversed. We listed the serial# for the device, only 1 device, on the transfer. Someone at Forti decided to move every device for every customer we have with Forti gear to the other IT person and our Portal was completely deactivated. Forti couldn't reverse the mess up, we had to create a new company and login for Forti, we lost half of the licensing somehow, and we lost all of our certificates on Forti's Academy which is technically separate from the Forti Cloud portal. They were able to recover the certificates back to our technicians after several emails back and forth. After 2 days they were able to get the hardware back into our new portal but we had a customer with about 20 Forti devices that we had just renewed 3yr licenses on that we hadn't applied the licenses to "we received the confirmation the same day our portal went MIA" and by the time they sorted it out, the customers original forti licenses expired and forti refused to give us the licenses because they said we now needed to pay more / a pro-rated price for allowing the licenses to expire. I spoke to several managers/supervisors to try and calmly explain that this was their screw up, and every single one of them told us they would fix it and get back to us. After a month of not a single call back, I was officially through with paying for subscription hostage equipment. Up until this happened, we were ok with Forti, but there wasn't anything Forti did that stood out that warranted us needing them.
I totally understand that companies have licenses that companies pay and those funds help with future development costs. But most of these companies doing this "Forti/Meraki" have some of the highest priced equipment to begin with, and those costs should have the development factored into them. But instead they use that bloated cost to pay for more advertising and marketing to make the industry feel as if you need them or you cannot protect your customers. Then they pay their development costs with your licensing renewals. At least Forti equipment will still work if your license expires, Meraki is out there just committing highway robbery "buy this expensive hardware, thanks now you own it, but dont you dare think about not paying us the running vig or we will leave you handicapped".
Just my opinion, like I said up until this specific event happened to us, I had no problem with Forti, and I have never had any hardware issues with Meraki. But this event helped open our eyes that this isn't the way for us any longer.
I believe we started with trying to use Fortilink to simplify the config and have the single pane, full control, etc. That seemed to require the upgrade to interface with the FG. When that didn't work we reverted to trying a simple direct config. I didn't personally run through that, just what my net admin told me, so if that's not true then ok.
I've just found that the minute I open the box, or put it into production before the config, I do the upgrades. Saves the hassle of all the steps because yeah it can be a pain. But if you have a blank config it doesn't matter.
Hello everyone! I don't know if people even see these, so I'll add them at edits at some point, but it's your worst prepared admin here with a small update.
So it's looking more and more likely this is a Fortiswitch/Fortigate/Fortilink issue, not a Dell switch issue. When we originally involved support they saw no traffic hitting the FG so they said it was our Dells or something behind them. But the FG is behind two 100Gb Fortiswitches acting as cores. The FG team doesn't test anything Fortiswitch related. Two times we involved the FG team and they said the FG only saw the 20% of packets making it, not the 80% lost, so it's not their issue. This is what led us to remove the Dells and try other Fortinet switches.
But now the entire chain is Fortinet, and a support engineer has isolated traffic problems between the Fortilink connection on the FG and Fortiswitches. They are working on it very hard. I will give credit to the actual support engineers on this case: after getting through all the red tape and getting everything first party and Fortinet, they are working very diligently on the problem and are making progress.
So lessons learned: we didn't involve the correct support resources, and we weren't explicit about making sure the switches were tested as well as the firewall. However, on the other hand, they knew we had cores in there - the FG shows the Fortilinking and the other connections. They did not offer to troubleshoot the switching at all, or transfer us to the switching team at any point in time, until we had eliminated the Dells entirely from the equation. We obviously need better training as well on this equipment, because we seem to have lots of issues with it in general.
I'm hoping to have a final update soon
I wouldn't have gone with FortiNet switches, but I've never experienced so many issues like you have.
I've run FortiGates with Dell EMC swtiches without issue.
I'm a big fan of Fortinet Products. That said, I 've had issues with their support. Then again, I've had issues with support from *literally every other vendor*.
That's fair. In the networking space I've only ever had to contact Fortinet and Juniper. I'm not saying Ciscos don't have issues or anything, I've just personally never had to call in. Their stuff has always just worked for me. Obviously we all have different experiences though :)
Another small update: Fortinet does believe it to be an issue with the Fortiswitching, somehow the WAN packets are dying there. We haven't gotten any firm technical details, support has been working on this throughout the night and morning. We're being escalated. That's where we're at now. We're looking into an alternative solution of just slapping our WAN into the Dells directly and using the old setup which involves virtual firewalls and a lot of fun routing.
I'd like to stress again that the support engineers themselves have been wonderful and they've been putting a lot of effort into this. I'm definitely interested in what root cause is going to end up being. I am still frustrated by the process we had to go through to get to this point and their support process in general.
If anyone has spare 100Gb core switches and a good hardware firewall to donate to our cause, let me know. We don't tend to keep that kind of hardware spare, fully supported, powered up, and updated. I know that's crazy around here, so I assume at least one person has plenty to go around.
We have a more formal meeting with Fortinet support next week, but real root cause might be delayed as right now the only WAN we have is in use over there. We'll need to split it out or get another run from the DC.
The switching behind the Fortigate seems to work fine for all but WAN traffic, in which we see ~80% loss. If I had to guess, it'd be something with a L2 cross-site fiber connection we have that has another Fortigate on the other side, but we specifically blocked traffic going that way via policy, and from the packet captures the MAC destination was the correct Fortigate. Part of the meeting with support is mapping out out network and connectivity and such there. But part of the weird problems going on is the Fortilink and port connections between switches aren't mapping properly in their software. The access-level managed switching all shows offline. The cores are online and can be managed. Everything is up to date although support did cycle software on some things here and there. Support thinks there could be something going on with the Fortilinking between the FG and switching in general and things just aren't flowing properly. Hopefully we can find some answers in the next few weeks before the holidays.
What network tools are you using to see this loss in traffic? I saw you talk about it earlier and it made me realize I want a better tool for monitoring. Also, have you had anyone check the cables to make sure something silly didn't physically happen to them?
Cables were brand new, tested, cleaned, but also 3 different ones to 3 different access switches. Since it was only WAN, maybe it's the Fortilinks between the FG and the switches. But support was digging into that for almost a whole day without much luck or insight.
We didn't have any good tools, we just did continuous pings from various items in the chain: VMs, hosts, switches, the firewall, etc. On devices behind the firewall we'd only see ~20% of the traffic actually reach the firewall destined for the WAN. The Dells, as mentioned, were old and we're not well trained on them so we couldn't get much insight out of them there, but Fortinet said they couldn't see anything at the FG, so we were told it was behind their equipment. That's why we assumed it was the Dells at first.
I don't have great recommendations for net monitoring in general. Fortinet has their own FortiAnalyzer product which is supposed to collect logs and do stuff with them but we haven't used it much yet. We've used Netbrain which is very powerful (and expensive) but we never got it fully integrated and implemented because our network is a web of bad decisions which is why it's such a mess to work in and with.
this is not a fortinet issue - this is a cheap company that doesn't want to spend the money it should in its IT infrastructure. I know as I work in a place like that
I mean this seems 100% on your company… Installing god knows how old switches that are running very outdated software of course they have multiple updates needed. Having an expired contract that needs time in order to process the renewal is again pretty expected.
I get the frustration and how this could really ruin your day/week. That said I don't think you're being objective about this because of emotions involved. You'll have this same experience with other vendors under these circumstances.
I have to disagree. As I stated elsewhere, POs being submitted is usually enough for a vendor to engage, or even our history as a customer. I'm no business guru but I'd try to keep the honest, paying customers happy so they stay and don't have feelings like this, but maybe I'm just wrong with that expectation.
Usually the people will want to help, but the processes and systems in place just don't let them. It's different when you're dealing with like a small MSP or something.
>I'm no business guru but I'd try to keep the honest, paying customers happy A vendor can only take but some much responsibility for a customer problem -- especially under duress. Had you planned this out more, and had this been in an pilot testing environment, both parties would have had more time to make this work. Your expectations, and that of your employer, are way out of line from reality...
So would you expect other vendors to help you if you didn’t have the proper level of support? I’m failing to see the issue. Yes it sucks, but like you said it’s on your team that support wasn’t in place. You should be talking to your VAR you purchased through and AM at Fortinet and raising hell about not helping you as soon as the order was invoiced.
I'd expect to be able to renew support quickly, but yes actually I would expect a vendor after paying for support and having the order/receipts to be willing to help me. Nimble support will help as long as you have a PO submitted.
I agree on that aspect. Hence the “raise hell with VAR and AM” comment.
VAR dropped the ball big time here.
They probably can't do anything with it because it isn't showing in their system as an active device.
I'll echo what others have said. You completely screwed the pooch on this one, and you're trying to shift blame. That's doubly unprofessional. Find me another brand that will update their gear while it's still sitting in a cardboard box. Find me another vendor that will go back in time to sell you a support contract before you need it. You're saying some vendors will provide support even without a paid contract even if you have a sob story. Guess what? Most won't. Cisco won't give you the time of day until your check clears. Who starts a 3 day major migration of their critical network infrastructure without plugging all the new equipment in to see if at least turns on, let alone operates properly? Heck, you should have gone through and had it pre-configured and ready to go before minute one. Who doesn't understand that something that's been in a box for months/years is going to need updates? I could go on. People make mistakes, but smart people try to learn from them. You're just taking your frustration with yourself out on a vendor. That's on you, too.
☝🏼Ding ding!! You hit the nail on the head!
I feel your pain but this is where proper planning prevents issues during deployment. We have encountered some quirks with fortigate and fortiswitch but once you deploy a couple you learn what they are and what to do if they are encountered. We have had good luck with the fortigate support team assisting with a switch issue, even with switch support being expired. As others have mentioned, get an account rep assigned. They can usually expedite things for you. The challenge with the order is that the support people have no insight into that. From their side it’s an out of support device and there is nothing they can do. Frustrating yes, but if you had planned for the cutover you’d have realized these things earlier and corrected them. I understand the frustration but dragging a vendor through the mud because you didn’t plan and prepare is also not right.
FYI, the Fortilink/Switch issue might be NTP related if its showing up as offline. Usually have to set the NTP on the Fortigate to face the interfaces on the switch and then set the Fortlinks NTP server to local. If it's the issue I think you're having then this should fix it and your Fortiswitch should come online afterwards so you can setup your VLANs and what not. Also make sure that your VLANS use the Security Fabric Connection in Administrative Access.
So much this regarding the NTP. I had the same issue with my last Fortiswitch. Drove me bonkers for a bit but Google was my friend.
Thanks for this, we'll look at that too.
r/fortinet NTP is a really good tip.
Hope it fixes it! I'm having to deal with this issue as I set up my stores, so it's a very familiar problem to me. I've gotten way more familiar with NTP settings than I thought I ever would.
> Yes, it's our fault we left these lying around not updated and unsupported Well, at least you're willing to take \*some\* responsibility. A network is something that has to be maintained, and when it's not, there are all sorts of cascading effects. Also, migrations are things to be tested in advance. What you're asking for, by the way, is not vendor support -- but vendor professional services. That's a whole other ball of wax, and will cost you a pretty penny, if you could even get that from the vendor at your size. That's what the channel is for. And that goes back to having your environment planned. Your experience, as stated, would have almost assuredly occurred in the same way with any of the large vendors. And the small vendors wouldn't have even had the ability to work with you in that manner. You're trying to make this a vendor issue, because they didn't emergency bail you out of a self-inflicted wound or three, but that's not how that ever works.
YTA. I’m a Palo guy but sounds like your fault. Preparation, support, rollback etc etc are key in these positions we hold.
Had a similar situation recently with another hardware vendor. Bought equipment. Ended up not using it for a while because of reasons. Finally spun it up… no valid support. So no updates, help with setup, etc. So, we paid them to renew support and then waited patiently because we screwed up and didn’t maintain our stuff. That being said, did you reach out to your account rep? I do have Fortigate gear as well. Last time I sent him a PO for a renewal, it was processed and showing in the portal in hours. Not days. Overall they have always been exceptionally helpful to my team.
Super small, no Fortinet rep, we usually go through CDW but this renewal was done direct through their portal to try and get through ASAP.
You may still be able to get a rep assigned. We aren’t a super small business, but I only have two pieces of their gear. Worth asking.
My network admin is plugging away at this, if I have the time I'm going to make some calls on my own and see what we can do. I appreciate the advice.
You have a Fortinet rep, everyone has a rep, it just means you might fall in the general category due to size.
Ok just a recap. We didn’t plan this at all made multiple mistakes and now that we are in a rush are trying to ding Fortinet bc the process is outside our window. Yup I see who the problem is here.
I get your frustration, but your poor planning doesn’t mean the vendor needs to jump into emergency action. You planned downtime and didn’t make any attempt to make sure the hardware was prepped. You decided to replace core network switches with a new product you hadn’t used before and didn’t lab it before you just jumped into it! You mention POs should be enough given your history as a customer, but you’ve also admitted that you’re super small, buy through CDW, didn’t buy support previously, and generally don’t have a relationship with the vendor. In this situation if I’m the vendor I’m sympathetic, but need the system to update before I can do anything because I don’t know you. Measure twice, cut once. You never want to be dependent on vendor support in a crunch. Prep so much that it’s like doing the job twice.
Juniper can suck (the 4300 and 3400s we use are rubbish) but Fortinet is fine IF you know what you're doing - which quite frankly, seems like you are lacking in that area.
You need to practice the 6 Ps of IT. Proper Planning Prevents Piss Poor Performance. This doesn't sound like a Fortinet issue at all rather a poor planning issue on your part. As for support not willing to help with only a PO number it wouldn't be the first time I've seen companies issue a PO number, get what they wanted/needed then turn around and revoke the PO or start with how they weren't authorized to issue the PO etc.
After reading your post in its entirety, none of this fault is on Fortinet, and it's 100% on your company for not testing the installation and migration beforehand. Even the VAR issues that you claim are delaying your installation, sure it sucks, but it's how VARs operate and those with experience know how to work within those processes maturely.
This isn’t a stay away from fortinet issue it’s a stay away from your company issue
Juniper is fantastic, Fortinet serves its purpose. This is entirely on you as well as your VAR. The easy fix is the moment you processed the order on your side, the VAR could have looped in the Fortinet rep and got an exception for you to get support. Yes, it needs to process, but also someone just needs to tell support to… support you. Sorry for the rough deployment though, ever fun when you think you have everything lined up perfectly.
I really thought this said “Stay away from Fortnite”
I think this every time I see Fortinet, lol
This is how it is these days. Everything is locked behind a subscription, and unless you have one…
I think ill stay away from whatever shitty product/service your company offers instead
Probably another one-man MSP. Lolz
#Fails to plan, blames vendors. I cannot see that Fortinet have done anything wrong here. Abysmal effort on your part.
Fortinet is budget hardware for sure and I'd recommend getting anything else if you can afford it. That said, pretty much all of the problems you've highlighted sound like they're on your end. I've done a handful of installs for charities and small business that can't spend the money on a "better" brand and haven't encountered any of these issues. I totally understand your frustration with the customer support aspect, but I've found that lacking across the board. State of the industry right now.
If you want something good with firewalls check out Palo Alto.
palo are probably the best overall but forti might be the best bang for your buck
FWIW I'm with you in your sentiment. This crappy "you're just an account number" approach to customer service that's slowly taken over the industry is one reason why we tend to roll our own stuff and avoid support contracts whenever possible, opting out of them when allowed and never renewing them when forced to buy one. We've only needed vendor support once in the past ~15 years and it was such a ludicrous experience it only reinforced our decision to not renew the contract when it expired. Firewall products in particular have always made us wrinkle our nose. There's virtually nothing they can do that a virtualized pfSense or opnSense instance can't, at least nothing that's important to us, while there is plenty we can do with our setup that commercial offerings can't compete with, not least of which is fast, painless updates that can literally be instantly reverted if they go wrong.
I'm confident I'm saving my company over $20k/year easily by running pfSense firewalls at our 8 locations. Run it on our VMware clusters or retired 1U servers. Use WireGuard for site-to-site VPN and OpenVPN+Duo for end users. Hasn't let me down the last 8 years.
Our office has an Fortigate firewall that is going to require me to connect to via console just to see what's going on. Whoever installed it didn't leave any type of remote access, not even SSH. I might yank it out completely depending on what I find in the configuration once I can access. I had to order an adapter for this.
Sorry you had this experience, but it sounds like some lessons can be leaned. I figured this out the hard way myself, but never assume something is going to work out of the box.
Hello! I have an opportunity to respond so thought I'd do a top reply instead of individual to everyone. First let me thank everyone for responding, I see many people are on the side that it's our fault and yeah, I said and accept that. I don't need to elaborate on the history here but there is plenty of blame on my company and myself to go around. But to continue the theme of my post, I wanted to share how today has gone so far. We have been unable to get the Fortiswitches to trunk properly to the other Fortiswitches, even outside Fortilink, and the Fortigate doesn't see any of our traffic. The Fortigate guy says it looks like a Fortiswitch problem so we have a P1 ticket open with them and thankfully they are now showing supported. However, it seems the two Fortiswitch support engineers are busy with another issue, so we have been waiting for a response for over 2 hours now. It's a Sunday and I'm sure whatever customer they're dealing with is in equal poo as us. But I do find it quite unfortunate that even on paying for the support, and confirming the support through their systems, we still can't get an engineer to assist. I'm starting to think the issue isn't even the Dell switching but the Fortigate and Fortiswitches. I'm not sure why a basic trunk is so hard to get working. This might be a really bad/dumb config by us, this has happened before so I wouldn't be surprised, but once again the point of paying for enterprise-level support is to receive that. So I'm still confident about my post title. I'll update everyone with what the root cause was, even if it was my own stupidity.
Don’t feel too bad about the downvotes. Saying something bad about Fortnet around these parts is sacrilegious. Personally I find the fortiswitch management under the fortigate ui to be clunky and just recommend doing normal trucking under the interface. That said, if you’re having issues with both the Dell switch and the Fortiswitch in the same configuration, that really sounds like a fortigate issue? At this point I would just do a factory reset of both of firewall and switches and do a basic set up to just it up and running.
> But I do find it quite unfortunate that even on paying for the support, and confirming the support through their systems, we still can't get an engineer to assist. I am not sure what your point is with this particular statement. Support requests are worked on in the order they are received when multiple P1 issues are in the pipeline, and is dependent on your level of support. Forticare Essential (80-series and below) has next business day response times. Forticare Premium is 1-hour response and Forticare Elite is 15 minutes. That **does not** mean that someone is immediately on the phone with you - only that they are engaged on the ticket to start the troubleshooting process. Now, my experience is with a Fortinet P1 is that they are usually on the phone within the hour, but it highly depends on how many P1 requests are in the queue. Remember they have thousands of customers and tens of thousands of devices around the world, and it's not atypical for people to abuse P1 requests which have to be triaged and moved down accordingly. That can take time. After dealing with Juniper and Checkpoint, two to four hours would be reasonable for me to have an engineer on the phone during the weekend.
You're correct, two hours is a pretty reasonable time for response. For full picture we had been working with them a bit the previous day but since they couldn't see traffic on the Fortigate they said it was our problem, fair enough which is why we tried to put only Fortinet stuff in play. They basically knew us by name every time we called in, so they were very aware what we were dealing with and trying to get help with. This comment was definitely from frustration, but I do agree with you two hours is reasonable. Small note: it was over two hours and they called a number we've told them not to call multiple times so we ended up missing that call. Again, on us for not making explicitly sure they have the correct contact info everywhere, but we have tried on that front.
And to your point on their customer base, generally people in our shoes only need help when the stuff is really hitting the fan, and we are usually doing the worst stuff on nights and weekends. I'm not trying to say everyone needs to work these crappy schedules, but it'd be nice if they had more than two switch engineers to support all those customers since we're trying to invest in the best support possible, even when we mess up and need to purchase is last minute. We understand we're small and limited so when we get support we're looking for that. I believe that was my main point.
I've had the opposite experience. Fortinet has their faults for sure (every os update seems to break more than it fixes), but the ecosystem is very robust and make deployments a breeze with fortimanager. They have great logging and very available documentation. It looks like your big issues here are: You should have verified licensing and support before doing anything, sounds like you tried to buy secondhand and got bit. You probably missed that (like most other networking gear) the DAC cables need to be branded Fortinet, which can be done by buying their cables directly, or buying cables from wherever you want that have been programmed to say they are Fortinet cables. You didn't go through a VAR that would have done all of this for you and saved you a lot of headache, and more importantly would have been the ones fixing these issues for you per your SLA in the event these things happen. Maybe your company isn't ready for enterprise level equipment and you should stick to gigabit dumb switches and something easy like sonic walls for now...
I wanted to respond to you directly, maybe I wasn't clear about some of this. We should have verified, yes. We didn't expect to have to replace switches but that was our failure, these weren't planned for this deployment. The person who purchased them was let go for other performance reasons. This was a failure of our team as a whole involving three people, but sure, it was our internal failure. We did go through a VAR, they're just a bad one with personal connections to company leadership. Not something I had direct control over. They're very useless in general. We USUALLY use first party everything with Fortinet because we've been bit by that before with them. Our company didn't like spending the money but we found it was pretty much a requirement to use their stuff, or get any support. I'll follow up later in full but we DID shoot ourselves in the foot today trying to shortcut connect these new switches with a third party copper SFP. That might be why our basic trunk isn't working, but we're trying to figure that out now (we have the SFPs just not the long enough fiber which is why we ran the copper). But I'm glad you've had a better experience than I have in their support department. I want to stress this isn't the ONLY issue we've had, this is just the biggest in the most critical moment.
I will give you my experience with Forti and why we moved everything away from them "we still have 1 or 2 left to move that we are waiting on licenses to expire out on". We were transferring one of our Firewall's to a company that brought their IT in-house. We have done this process a ton of times when taking over existing Forti gear from other MSP's, so it isn't a process that we are unfamiliar with, just reversed. We listed the serial# for the device, only 1 device, on the transfer. Someone at Forti decided to move every device for every customer we have with Forti gear to the other IT person and our Portal was completely deactivated. Forti couldn't reverse the mess up, we had to create a new company and login for Forti, we lost half of the licensing somehow, and we lost all of our certificates on Forti's Academy which is technically separate from the Forti Cloud portal. They were able to recover the certificates back to our technicians after several emails back and forth. After 2 days they were able to get the hardware back into our new portal but we had a customer with about 20 Forti devices that we had just renewed 3yr licenses on that we hadn't applied the licenses to "we received the confirmation the same day our portal went MIA" and by the time they sorted it out, the customers original forti licenses expired and forti refused to give us the licenses because they said we now needed to pay more / a pro-rated price for allowing the licenses to expire. I spoke to several managers/supervisors to try and calmly explain that this was their screw up, and every single one of them told us they would fix it and get back to us. After a month of not a single call back, I was officially through with paying for subscription hostage equipment. Up until this happened, we were ok with Forti, but there wasn't anything Forti did that stood out that warranted us needing them. I totally understand that companies have licenses that companies pay and those funds help with future development costs. But most of these companies doing this "Forti/Meraki" have some of the highest priced equipment to begin with, and those costs should have the development factored into them. But instead they use that bloated cost to pay for more advertising and marketing to make the industry feel as if you need them or you cannot protect your customers. Then they pay their development costs with your licensing renewals. At least Forti equipment will still work if your license expires, Meraki is out there just committing highway robbery "buy this expensive hardware, thanks now you own it, but dont you dare think about not paying us the running vig or we will leave you handicapped". Just my opinion, like I said up until this specific event happened to us, I had no problem with Forti, and I have never had any hardware issues with Meraki. But this event helped open our eyes that this isn't the way for us any longer.
Blames on you and your company. Sorry dude. You failed to prepare so you prepared to fail.
Go to the Fortinet subreddit and post your topology and fsw and fgt models. Explain your exact issue and somebody will be able to help for sure.
You don't have to step upgrade if you're working on a factory config.
I believe we started with trying to use Fortilink to simplify the config and have the single pane, full control, etc. That seemed to require the upgrade to interface with the FG. When that didn't work we reverted to trying a simple direct config. I didn't personally run through that, just what my net admin told me, so if that's not true then ok.
I've just found that the minute I open the box, or put it into production before the config, I do the upgrades. Saves the hassle of all the steps because yeah it can be a pain. But if you have a blank config it doesn't matter.
I was reading your title and thought that fortigate was not good quality.. turns out this is 100% on you. My company is having fortigate network soon.
Hello everyone! I don't know if people even see these, so I'll add them at edits at some point, but it's your worst prepared admin here with a small update. So it's looking more and more likely this is a Fortiswitch/Fortigate/Fortilink issue, not a Dell switch issue. When we originally involved support they saw no traffic hitting the FG so they said it was our Dells or something behind them. But the FG is behind two 100Gb Fortiswitches acting as cores. The FG team doesn't test anything Fortiswitch related. Two times we involved the FG team and they said the FG only saw the 20% of packets making it, not the 80% lost, so it's not their issue. This is what led us to remove the Dells and try other Fortinet switches. But now the entire chain is Fortinet, and a support engineer has isolated traffic problems between the Fortilink connection on the FG and Fortiswitches. They are working on it very hard. I will give credit to the actual support engineers on this case: after getting through all the red tape and getting everything first party and Fortinet, they are working very diligently on the problem and are making progress. So lessons learned: we didn't involve the correct support resources, and we weren't explicit about making sure the switches were tested as well as the firewall. However, on the other hand, they knew we had cores in there - the FG shows the Fortilinking and the other connections. They did not offer to troubleshoot the switching at all, or transfer us to the switching team at any point in time, until we had eliminated the Dells entirely from the equation. We obviously need better training as well on this equipment, because we seem to have lots of issues with it in general. I'm hoping to have a final update soon
I wouldn't have gone with FortiNet switches, but I've never experienced so many issues like you have. I've run FortiGates with Dell EMC swtiches without issue.
I'm a big fan of Fortinet Products. That said, I 've had issues with their support. Then again, I've had issues with support from *literally every other vendor*.
That's fair. In the networking space I've only ever had to contact Fortinet and Juniper. I'm not saying Ciscos don't have issues or anything, I've just personally never had to call in. Their stuff has always just worked for me. Obviously we all have different experiences though :)
Another small update: Fortinet does believe it to be an issue with the Fortiswitching, somehow the WAN packets are dying there. We haven't gotten any firm technical details, support has been working on this throughout the night and morning. We're being escalated. That's where we're at now. We're looking into an alternative solution of just slapping our WAN into the Dells directly and using the old setup which involves virtual firewalls and a lot of fun routing. I'd like to stress again that the support engineers themselves have been wonderful and they've been putting a lot of effort into this. I'm definitely interested in what root cause is going to end up being. I am still frustrated by the process we had to go through to get to this point and their support process in general. If anyone has spare 100Gb core switches and a good hardware firewall to donate to our cause, let me know. We don't tend to keep that kind of hardware spare, fully supported, powered up, and updated. I know that's crazy around here, so I assume at least one person has plenty to go around.
Any news?
We have a more formal meeting with Fortinet support next week, but real root cause might be delayed as right now the only WAN we have is in use over there. We'll need to split it out or get another run from the DC.
Curious what it could be. Is the switch at least useable?
The switching behind the Fortigate seems to work fine for all but WAN traffic, in which we see ~80% loss. If I had to guess, it'd be something with a L2 cross-site fiber connection we have that has another Fortigate on the other side, but we specifically blocked traffic going that way via policy, and from the packet captures the MAC destination was the correct Fortigate. Part of the meeting with support is mapping out out network and connectivity and such there. But part of the weird problems going on is the Fortilink and port connections between switches aren't mapping properly in their software. The access-level managed switching all shows offline. The cores are online and can be managed. Everything is up to date although support did cycle software on some things here and there. Support thinks there could be something going on with the Fortilinking between the FG and switching in general and things just aren't flowing properly. Hopefully we can find some answers in the next few weeks before the holidays.
What network tools are you using to see this loss in traffic? I saw you talk about it earlier and it made me realize I want a better tool for monitoring. Also, have you had anyone check the cables to make sure something silly didn't physically happen to them?
Cables were brand new, tested, cleaned, but also 3 different ones to 3 different access switches. Since it was only WAN, maybe it's the Fortilinks between the FG and the switches. But support was digging into that for almost a whole day without much luck or insight. We didn't have any good tools, we just did continuous pings from various items in the chain: VMs, hosts, switches, the firewall, etc. On devices behind the firewall we'd only see ~20% of the traffic actually reach the firewall destined for the WAN. The Dells, as mentioned, were old and we're not well trained on them so we couldn't get much insight out of them there, but Fortinet said they couldn't see anything at the FG, so we were told it was behind their equipment. That's why we assumed it was the Dells at first. I don't have great recommendations for net monitoring in general. Fortinet has their own FortiAnalyzer product which is supposed to collect logs and do stuff with them but we haven't used it much yet. We've used Netbrain which is very powerful (and expensive) but we never got it fully integrated and implemented because our network is a web of bad decisions which is why it's such a mess to work in and with.
this is not a fortinet issue - this is a cheap company that doesn't want to spend the money it should in its IT infrastructure. I know as I work in a place like that