Managing Elasticsearch Reindex at Scale: Performance, Reliability, and Observability

Editor’s Note: This is the fourth post in a series exploring how Palantir customizes infrastructure software for reliable operation at scale. The following is a guest contribution to the Foundations series from the Gotham Core Platform organization, which builds and maintains the bedrock for mission-critical applications within the Gotham ecosystem. This blog post by Kevin Liang, a backend developer based in CA, highlights the design considerations and improvements made to the Elasticsearch reindex machinery — which broadly aims to provide an easy-to-use, performant, reliable, and observable w