Real-time Feature Serving for Online Inference

Speaker: Amy Nguyen

Real-time machine learning depends on features that fundamentally cannot be pre-computed. Users expect their next recommendation to be based on their most recent behavior. Detection of fraudulent behavior or acute diseases such as sepsis depend on data from the last few seconds. Exhaustive retrieval of data from external APIs can be cost-prohibitive or logistically impossible. Modern feature stores must grapple with these limitations by truly computing features just-in-time. In this talk, we'll look under the hood at how Chalk built its just-in-time online feature store to enable realistic machine learning use cases while still serving requests in under 5 milliseconds.