My guess was, as you mentioned in your article, that common questions the machines have already been trained on are solvable because they already know the logic. I did want to give it a go though. Because it means machines are learning information of which a subset is how to work through certain types of problems.

I'll definitely try to trick it though, just to prove to myself that their limits are quite close if you look for them.