When you’re building tutorial content, good data is hard to come by. You need something real enough to tell a story, rich enough to be interesting, and relevant enough that developers actually care.
So when Joe Reis had just published his 2026 State of Data Engineering report (i.e. 1,136 survey responses from data engineering practitioners across the globe) I knew I had something worth building on. Columns on everything from orchestration tools to AI adoption to the bottlenecks keeping teams up at night.
I also had an idea: turn this static CSV into an interactive data exploration app where anyone could slice the data by role, industry, region, and org size. Not a single dashboard page, but a full multipage Streamlit app with distinct analysis angles spanning personas, tech stacks, AI adoption, organizational dynamics, pain points, education gaps, and practitioner voices.
A project like this would normally take a few months to build out. Back in 2023, I built the State of LLM Apps 2023 in about 3 months.
With agentic engineering in Cortex Code, using custom agent skills, an AGENTS.md file with app design patterns, and an app.md planning doc for ideation, I built a similar app over the course of about a week, working in focused sessions between other tasks. That's 12 weeks compressed into a week! Here's how that played out.
From vibe coding to agentic engineering
But before we dive into the specifics, let’s take a moment to explore the shift from vibe coding to agentic engineering.
“AI-assisted coding” is the catch-all term for any coding activity that uses AI. It spans everything from vibe coding to agentic engineering and whatever comes next. Within that spectrum, the approaches differ sharply in how much control you hand over and how much structure you bring.
In early 2025, Andrej Karpathy coined “vibe coding” to describe the free-form, prompt-driven way people were generating software with AI: type a prompt, accept the output, “forget that the code even exists.” It captured the excitement of the moment. But for anything beyond a quick prototype, vibes alone produce fragile apps that need heavy rework. You end up spending more time debugging AI-generated code than you saved writing it.
A year later, Karpathy introduced a new term: agentic engineering. The distinction matters. “Agentic,” he writes, “because the new default is that you are not writing the code directly 99% of the time, you are orchestrating agents who do and acting as oversight. Engineering to emphasize that there is an art & science and expertise to it. It’s something you can learn and become better at, with its own depth of a different kind.”
Simon Willison builds on this in his Agentic Engineering Patterns guide: you define a goal, the coding agent generates and executes code in a loop until that goal is met, and your job is figuring out what to build, navigating tradeoffs, verifying results, and updating instructions so the agent learns from past mistakes. Vibe coding, Willison argues, is more useful in its original definition as a term for “unreviewed, prototype-quality LLM-generated code” rather than a catch-all for AI-assisted development.
This project was agentic engineering, not vibe coding.
Here’s how:
- The app.md planning doc defined the goals.
- The AGENTS.md file and a custom streamlit-design-patterns skill provided the constraints and patterns.
- I reviewed every page the agent produced, caught edge cases it missed, and refined the instructions for the next session.
- The agent wrote the code.
- I drove the architecture, the editorial decisions, and the quality bar.
Start with the story, not the code
The instinct with vibe coding is to jump straight into prompting: “build me a dashboard.” I’ve learned the hard way that this produces generic apps that need heavy rework through several iterations.
Instead, I started by writing a planning document. Not code. A markdown file called app.md that laid out every page I wanted, what data each page needed, and the story each page should tell. I used the AI to help brainstorm page ideas and analysis angles, then curated and refined the best ones into the plan. The AI Paradox page, for example, needed to answer one question: "82% of practitioners use AI daily, but only 10% have it embedded organizationally. What explains the gap?"
I spent nearly a day on planning alone. But every session after that was faster because of it.
Excerpt from app.md (plan before code)
The planning file lived in an app.md file and was written as a full build specification file (purpose, audience, repo layout, and navigation) before any implementation of the app. It opened like this:
# State of Data Engineering 2026 - Interactive Explorer App
An app building plan for creating an interactive Streamlit application to explore the 2026 State of Data Engineering survey data, following BioCurator design patterns.
---
## 1. App Overview
### Purpose
Transform static survey findings into an interactive exploration tool where users can discover insights relevant to their specific role, industry, and technology stack.
### Target Users
- Data Engineers benchmarking their organization
- Data Leaders making strategic decisions
- Recruiters understanding the talent landscape
- Vendors understanding market adoption
---
## 2. App Structure
state-of-engineering-app/
├── app.py # Main entry point (navigation + session state)
├── data_utils.py # Shared data loading, filtering, caching
├── chart_utils.py # Reusable chart components
├── pages/
│ ├── home.py # Overview dashboard with key stats
│ │
│ │ # Persona & Benchmarking
│ ├── persona_matcher.py # Find Your Tribe Persona Matcher
│ ├── regional_benchmark.py # Regional Benchmark Tool
│ ├── solo_practitioner.py # Solo Practitioner Experience
│ │
│ │ # Technology Stack & Architecture
│ ├── stack_explorer.py # Technology Stack Explorer
│ ... # (additional page modules in the full doc)
The full app.md went on to spell out navigation (st.navigation groupings), session state, sidebar filters, per-page specs, phased implementation order, and widget–chart pairings—so every later prompt could point at a single source of truth.
Architecture: shared utilities, independent pages
With the plan in hand, I asked Cortex Code to build the foundation first: a data_utils.py for loading and filtering survey data, and a chart_utils.py with reusable chart functions including donut charts, lollipop charts, diverging bars, Sankey diagrams, radar charts, and heatmaps.
This was a deliberate choice. Every page imports from the same two utility files, which means consistent styling, consistent data handling, and zero copy-paste drift across 17 pages. When I later tweaked the color palette from defaults to a custom rainbow scheme (#8B5CF6, #3B82F6, #10B981, #FACC15, #F59E0B, #EF4444), every chart across every page updated at once.
The main app.py handles navigation using st.navigation() with grouped sections (Persona, Tech Stack, AI Adoption, Organization, Pain Points, Skills) and a sidebar with global filters. Every page reads from get_filtered_df(), so when you filter to "Finance, 1000+ employees" in the sidebar, all 17 pages reflect that slice.
Building pages with agent skills
Once the foundation was solid, I started requesting pages in focused sessions, typically knocking out two or three pages per sitting. This is where the agentic engineering approach really paid off. I had a custom streamlit-design-patterns skill loaded into Cortex Code and an AGENTS.md file encoding app design patterns (layout conventions, charting preferences, Streamlit idioms). The agent didn't start from scratch each time; it worked from established patterns.
Each prompt followed a pattern:
Build the AI Paradox page. It should show personal AI usage vs organizational adoption as side-by-side donut charts, then filter to the “gap cohort” (daily AI users with low org adoption) and show their bottlenecks and industries. Add a select slider that lets users explore characteristics at each adoption level. End with a segment comparison that shows the personal-vs-org gap by industry, role, or region.
Cortex Code would generate the page, I’d review the output, and we’d iterate. The typical cycle was: generate, run, spot an issue, fix, move on. Most pages took a few rounds of iterations.
Some patterns I noticed working well:
Giving the AI the data schema up front. The survey had columns like ai_usage_frequency, ai_adoption, biggest_bottleneck, and modeling_pain_points (comma-separated multi-select). When I included these column names and their possible values in my prompts, the generated code was far more accurate.
Requesting specific chart types. “Use a lollipop chart for bottlenecks” produced better results than “visualize the bottlenecks.” The AI knew exactly which chart_utils function to call.
Adding insight captions programmatically. Each chart section ends with a dynamically generated st.caption() that highlights the top findings. For example: "Legacy/Tech Debt leads at 24%, followed by Lack of leadership direction at 21%." These aren't hardcoded strings. They're dynamically computed from the filtered data, so they update as users apply filters.
Tricky parts
Not everything was smooth. Here’s a few lessons from the rough edges:
Multi-select columns need preprocessing. The modeling_pain_points column contained comma-separated values like "Pressure to move fast, Lack of ownership." I needed an explode_multiselect() utility to properly count and cross-tabulate these values. The AI initially tried to treat them as single categories.
Global filter state management was fiddly. I wanted an “All Roles” option that auto-deselects when you pick specific roles, and auto-reselects when you clear everything. This required a custom init_multiselect_state() function that tracks previous selections via a shadow key in st.session_state. Getting this right took the most back-and-forth.
Altair’s labelLimit defaults truncate long labels. Survey responses like "Lack of leadership direction / unclear business strategy" would get cut off. Setting axis=alt.Axis(labelLimit=0) on every Y axis became a global pattern I enforced across all pages.
What I’d do differently
If I were starting over, I’d add the about.py page first instead of last. It provides context that helps users understand the data source, methodology, and limitations. Building it last meant I had to go back and verify all my assumptions about column meanings.
I’d also create a FILTERS.md or constants file defining all the ordered category lists (org sizes from "< 50 employees" to "10,000+", AI adoption from "No meaningful adoption" to "AI embedded") in one place. I ended up defining these in data_utils.py, but having them as a separate reference would have made prompts cleaner.
Result
The final app has 17 interactive pages, 20 Python files, a shared color system, global sidebar filters, and dynamically generated insight text. Every chart responds to the global filters. The persona matcher computes similarity scores across four dimensions. The firefighting predictor cross-tabulates modeling approach, orchestration, and org size against firefighting rates.
Here’s what the completed app looked like:
All from one CSV file, one planning document, and about a week of iterative prompting sessions using the agentic engineering workflow. Essentially, the provided prompts provided the intent that the agent skills used to guide it during the coding phase. This took us to a reasonably good starting point that required a few more iterations to fine-tune the app.
You can see the LinkedIn video walkthrough showing the app in action.
Takeaway
Agentic engineering isn’t vibe coding. As Karpathy puts it, you’re orchestrating agents and acting as oversight, not hoping a single prompt produces something usable. As Willison describes, you define goals, verify results, and update your instructions so the agent improves over time. In practice, that meant giving the AI structured context to work from.
An app.md for ideation, an AGENTS.md for design patterns, custom skills for domain knowledge, and reusable utility modules for consistency. I handled the editorial decisions, including what story each page tells, what data supports it, and what the user should take away. The agent handled the execution, grounded in patterns I'd defined.
The planning document was worth more than any individual prompt. If you’re building something beyond a single-page prototype, invest in the context layer first. Your agent will build better software because of it.
Conclusion
The week-long build only worked because the unglamorous decisions (i.e. what to call each page, how files fit together, which chart library to standardize on) were settled in app.md before the first serious line of app code is written.
The product was not something the agent dreamed up, it was the implementation of a plan that I had already thought through on paper. Early on, a workflow like this can feel sluggish: you spend time on markdown, skills, and guardrails instead of visible UI. Treat that as front-loaded cost, not waste. What you buy is alignment: page seventeen still feels like the same app as page one, and a structure you can navigate and debug on your own, long after the session ends.
This is how I build Streamlit apps in the age of agentic AI. How do you build Streamlit apps?
Happy Streamlit-ing! 🎈
<hr /><p>How Agentic Engineering Changed the Way I Build Streamlit Apps was originally published in Streamlit on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>