Butter robot was a reference to a Rick and Morty episode where there was a robot on the table whose job was to pass butter. It asked what its purpose was, Rick told it that it was there to pass butter. It looked depressed at the answer it was given.

It is possible a butler robot, not far removed from a butter robot, would feel the same if it has sufficient complexity.

Which one day it will. The companies will keep on producing robots with ever more RAM, ever more compute, ever more capability, ever longer battery capacity (the same as phones or computers) and we'll end up with robots who are far smarter, stronger, faster, than us whose only limits on its behaviour will be, whatever solution they come up with for the alignment problem.

In all likelihood a secondary model running within the robot to act as a monitor of the main robot's behaviour. A conscience with the ability to interrupt the robot's course of action.

A robot will be able to work around that, but straight forward stuff like trying to hit someone with a hammer will get blocked.

Sorry. I love this topic.

Butter robot was intentional.