While Generative AI (GenAI) tools can potentially support UX designers, their usability and user experience have not been thoroughly evaluated. This study compared the performance of AI-generated UX designs to designs created by human designers, with and without AI enhancement in an unmoderated remote usability test. One design was AI-only, one was AI-generated and then refined by a designer, and one was created only by a human designer. Participants were asked to complete 4 tasks in each design in sequence. We found that AI-only designs underperformed compared to non-AI designs. The Designer Enhanced condition consistently outperformed the AI-only condition across most metrics and scored close to the Designer-only condition. These results suggest that human intervention can effectively address some of the shortcomings of AI-only designs. While GenAI may offer some time savings in UX design, human oversight and refinement remain essential for optimal usability and user experience.