Advanced machine learning and AI strike me as deterministic finite state machines with an error term. Properly tuned, the same input should produce much the same output every time. Like Newtonian physics, they can be complicated, but deconstructible into simple components. Like Newtonian physics, as well, they are highly accurate and reliable within their domain.
What happens when we start applying the same tools to complex systems? What are the complex systems of importance that refuse to yield identical results with identical inputs? Health in its totality (not just diagnosing specific disorders), political stability, climate change are a few to come to mind. Our tools may appear to work1 but then fail spectacularly. Now would be a good time to start worrying about this class of problems.
Value at risk (VAR) was a quantitative method to estimate risk exposure to owning financial assets. It was supposed to be a conservative check on the portfolio composition. It failed spectacularly in 2008 when implicit and explicit assumptions about the financial environment changed, and the interconnectedness of financial institutions around the world had been overlooked in favor of a unitary view of The Market.↩︎