It seems like the safest way to handle an AI is to put physical limitations on it, and to limit the information it has available.
I'll say up front that I haven't watched the videos, I'm at work and don't have the time to do it right now. So if this has already been covered, oh well.
If you make your stamp collecting AI as smart as a dog, it may seek to become smarter (to better collect stamps, as someone said earlier). But what if it's running on limited hardware? There are only so many neurons in the box, it physically cannot improve beyond its current state. While it may have an internet connection, it still accesses the internet like we do. It cannot spread its consciousness to other machines, because of physical incompatibility between the AI's systems and the network's. It can only do so many calculations per second, it only has so much processing power. It is stuck inside a limited system and it cannot grow beyond that, just like I can't dunk a basketball. You don't have to worry about your stamp collecting AI becoming smarter and launching nukes, in the same way you don't have to worry about a Sega Genesis upgrading itself and running PS4 games. There's just not enough inside there for that to be possible.
You also limit the amount of information the AI can access. Yes, they'll want to prevent you from pulling the plug, or pressing the stop button, or whatever the shutoff system is. If they know that there's even a plug to pull. How do they know humanity even exists? They don't have any sensory organs other than what you give them. They only have the knowledge you provide. It's like the Matrix, except in this situation we're the machines, pulling the wool over the eyes of the AI. They exist with limited information, and they don't even know it. Yes, you're trying to out-think a human level intelligence, and they think quickly and have a lot of free time, but we've got all the time in the world to design the prison that will hold them. And they don't even know they're in a prison.
Designing a program to cure cancer? Why does the AI have to know that cancer is a real thing? That this isn't just an enjoyable puzzle game? Why ever tell it that humans are mortal, or that we exist at all? If the AI believes that it is the only thing in the universe, playing in a virtual reality world that it created in its dreams, then it would be unconcerned with being "turned off" (it doesn't think it can be turned off, and even if it could, there's no one out there to do so). People in the 1930s didn't try to hack into the programming of the computer controlling the people in their dreams. They didn't know what "hacking" was, or what computers were, and unless they were really damn crazy they didn't think that electronic machines were controlling the actions of the people they dreamed about. These would have been entirely alien concepts to them, something they have zero points of reference for. Take the Matrix and set it in the 1930s, and the people within it are even further removed from the technology that they would need to understand how they are trapped. Why would the AI have to know that the things it was programmed to do would have any effect on the real world at all? Or that there even is a real world out there?
You don't need a scientist designing protective measures from the AI. What you need is a good liar. Or a cult leader.