The more I thought about the fact the card could output 625 & 525 the more convinced I was that your setup mwas viable for 405.
Anyway, well done.
Here are some "rules" that I stick to for a successful outcome.
Always make sure the PC boots up with VGA screens on both ports showing the extended desktop.
Then switch port 2 to the modulator
Only then activate the modeline ensuring you have the secondary or Video1 destination set on the dropdown menu
Also I found that if several modelines are listed in the application, things didn't always behave as expected, so I only have the 405 one in the list.
What modulator are you using?
I'm using this one
http://www.earlytelevision.org/405_modulator.html
I'm really impressed with the performance, both sound and vision, in fact I am building a second one now, which will have a tidier construction.
You may like to consider downloading Vidblaster X Trail
https://www.youtube.com/watch?v=ahzdx4KgWBg