Skip to content
NeuralForge

02Learning paths

Four routes through AI engineering.

Each AI developer learning path orders programs and projects into a sequence with clear prerequisites and milestones. All paths share the same foundations, so switching later costs little.

01

AI Application Engineer

Build user-facing applications powered by LLMs, from the first API call to a deployed, evaluated service.

Prerequisites

  • Comfortable writing Python; TypeScript experience is helpful for front-end exercises
  • Experience building or consuming REST APIs
  • Working knowledge of Git and the command line

Suggested pace: Approx. 5–6 months part-time (suggested)

View the AI Application Engineer path
fig. — structured outputIllustrative

input · support email

“Hi — I was charged twice for the annual plan on my team account this morning. Can someone refund the duplicate? Order 48213.”

output · TicketTriage

{

"category": "billing",

"priority": "high",

"order_id": "48213",

"needs_human": true

}

  • ✓ valid JSON
  • ✓ schema v3
  • ✓ enum values
  • retries: 0 / 2

Stages

  1. 01

    LLM Fundamentals

    Tokens and context windows · Calling a hosted model API or an open-weight model

  2. 02

    Prompting and Structured Outputs

    Prompt structure and templates · JSON Schema and Pydantic models

  3. 03

    Retrieval and Tool Use

    Chunking and embeddings · Vector search with pgvector

  4. 04

    Application Layer

    FastAPI endpoints with Pydantic models · Streaming responses to a TypeScript front end

  5. 05

    Evaluate and Ship

    Test sets and regression checks with pytest · Docker packaging and configuration

02

RAG & Knowledge Systems

Build searchable, grounded knowledge assistants that answer from your documents and show their sources.

Prerequisites

  • Comfortable writing Python
  • Basic SQL and familiarity with PostgreSQL
  • Experience calling a model API, or completion of the LLM Foundations program

Suggested pace: Approx. 3–4 months part-time (suggested)

View the RAG & Knowledge Systems path
fig. — retrieval-augmented generation

Ingestion · offline

  1. documents/

    PDF, Markdown, HTML

  2. parse + clean

    keep headings & ids

  3. chunk

    structure-aware, overlap

  4. embed

    vector per chunk

  5. index

    vectors + metadata

shared index

Query · online

  1. question

    user input

  2. retrieve top-k

    hybrid: vector + keyword

  3. rerank

    keep the best 4

  4. generate

    answer only from sources

  5. answer + [1][2]

    citations to chunks

Ingestion runs offline when documents change; the query path runs on every question.

Stages

  1. 01

    Model and Embedding Basics

    Tokens and context budgets · What embeddings represent

  2. 02

    Ingestion and Chunking

    Parsing PDFs, HTML, and Markdown · Preserving metadata and structure

  3. 03

    Search and Ranking

    Vector search and indexes · BM25 keyword search

  4. 04

    Grounded Answers

    Context assembly within a token budget · Prompting for grounded answers

  5. 05

    Evaluation and Tuning

    Recall@k and MRR · Groundedness and citation-accuracy checks

03

AI Agents & Workflows

Design tool-using, multi-step systems with explicit state, guardrails, and human oversight.

Prerequisites

  • Strong Python, including async code and type hints
  • Experience with structured outputs from a model API
  • Basic understanding of retrieval and embeddings
  • Comfort writing tests with pytest

Suggested pace: Approx. 4–5 months part-time (suggested)

View the AI Agents & Workflows path
fig. — bounded agent loop
  1. 1

    plan

    next action from goal + state

  2. 2

    select tool

    allow-listed registry only

  3. 3

    validate args

    schema check before running

  4. 4

    execute

    timeout + idempotency key

  5. 5

    observe

    append result to state

  6. loop while goal not met

Exits

✓ respond

goal met, cite tool results

⏸ ask a human

approval required for write actions

✕ fail safely

step limit (8) or invalid tool call

state = { step: 3/8, tools: ["search_orders"], pending_approval: false }
Every iteration either makes progress, finishes, or exits through a defined failure path.

Stages

  1. 01

    Tool Calling

    Tool schemas with JSON Schema and Pydantic · Argument validation

  2. 02

    Workflows and Orchestration

    Prompt chaining and routing · Parallel calls

  3. 03

    State and Memory

    Explicit agent state · Conversation summarization

  4. 04

    Guardrails and Oversight

    Input and output validation · Least-privilege tool permissions

  5. 05

    Testing Agents

    Scenario suites with mocked tools · Trajectory evaluation

04

LLMOps & Evaluation

Test, observe, and improve AI systems in production with evaluation suites, tracing, and monitoring.

Prerequisites

  • Experience building at least one LLM-powered feature
  • Comfortable writing Python and tests with pytest
  • Working knowledge of Docker and web service deployment
  • Basic familiarity with logging and metrics

Suggested pace: Approx. 3–4 months part-time (suggested)

View the LLMOps & Evaluation path
fig. — eval pass rate by prompt versionIllustrative
Eval pass rate by prompt version (illustrative)Groundedness rises from 62% at v1 to 89% at v6, dipping at v4. Format validity rises from 81% to 99%. The release threshold is 85%.50%60%70%80%90%100%release threshold 85%v1v2v3v4v5v6prompt versionregression
  • groundedness
  • format validity
Illustrative demo data on a fixed 120-case test set. v4 shows a groundedness regression caught before release.

Stages

  1. 01

    Evaluation Foundations

    Defining task-specific quality criteria · Building and versioning test sets

  2. 02

    Scoring and Failure Analysis

    Groundedness checks · LLM-as-judge rubrics and their biases

  3. 03

    Regression Testing

    pytest-based evaluation suites · Thresholds and flaky results

  4. 04

    Observability and Cost Control

    Tracing with OpenTelemetry · Token usage and cost per request

  5. 05

    Production Monitoring and Security

    Online quality signals and user feedback · Dashboards and alerts

Move from experimenting with AI to engineering it.

Build the foundations, projects, and habits needed to design more dependable AI applications.

Explore ProgramsBrowse Learning Paths