Infinite loops and doomed machines

CRANK

Remember that cascading failure post from a couple of weeks ago? I have another story about that same pseudo-database making life interesting. It has to do with the ability for nodes to have "parents" and "children".It seems that one fine afternoon, someone was messing around with this system, and they were adding nodes, deleting other ones, and generally changing the relationships between them. Somehow, they managed to violate a certain invariant, which is that you shall not introduce a loop into the graph.Yep, that's right. Somehow, one or more of the nodes in the system became its own ancestor. I don't think it was a direct "I am a parent of myself"/"I am my own child" relationship, but instead probably happened with some number of intermediate links.Exactly how it happened isn't super important, because once it did, things got rather exciting. There was this helper program which was supposed to pay attention to changes like this and then keep a copy of the updated graph around for…