Trade with Villager won't open

Whenever I try to open the trade menu with a villager (with a profession of course), the menu pops up for a second and then disappears. Therefore making it impossible for me to trade. I can trade with the wandering traveler but not with my villagers that already exist.