Nah, you're not crazy! Just DYING to get there...at any cost!
I went to WDW while I was 8 months pregnant and although I loved being away, I was MISERABLE the entire time. I got sciatica really bad while there and had to rent a wheel chair most days. I was pretty good up until the trip...I thought I could handle it, so I never cancelled my plans.
It was SO MUCH WALKING and probably caused SO much strain at that point in my pregnancy. I went while pregnant with my 2nd and was only almost 5 months along and WHAT A DIFFERENCE!
Night and day, really. What about going when the baby comes and is just a few months old? Then you will have the joy of seeing BOTH of your children at WDW! Genavieve took her first trip at 4 months old. I loved that trip!
Whatever you decide, good luck and have a blast!