Note on Cloud Rats and Software Roaches via Idle Words

Remember that death is the best form of reinforcement learning. A scenario where mutable rogue software exists, and 99.9% of it is successfully caught, is one where we’re guaranteed to get highly capable, persistent software pests in short order. These pests will learn to evade whatever controls we or our AI masters try to impose on them. And it’s not clear that AI, even assuming superhuman coding ability, will be smart enough to defeat them.

FROM:

Idle Words

Cloud Rats and Software Roaches

Source