← Writing

· 10 min read

Building a Skill-Gap Analyzer That Runs Entirely on My Laptop

How I built an offline workforce intelligence platform with Mistral 7B, FAISS and Streamlit on a 16 GB laptop: no GPU, no cloud APIs, no data leaving the machine.

Illustrative image

For my MBA research project, I wanted to answer a question I deal with every day in Learning & Development: how do you know which skills your people are missing, and whether you even have training for them?

Most L&D teams answer that with gut feel, last year’s training calendar or whoever asked loudest. I wanted a data-driven answer, and I had one hard constraint: everything had to run offline. Employee skill profiles, career goals and capability gaps are sensitive data. I didn’t want any of it going to a cloud API.

The result was the Strategic Workforce Intelligence Platform, which has two parts:

  • An HR command center that shows market skill demand, salary benchmarks and where the internal course catalog has gaps.
  • An employee skill-gap analyzer. You enter your current role, your target role and your skills. It works out what you’re missing and recommends courses from the internal catalog. All of it runs on an 8th-gen Intel i7 laptop with 16 GB of RAM. There’s no GPU and no internet dependency, and the whole system fits in under 10 GB.

Here’s how I built it.

The architecture: reasoning, retrieval, analytics

I split the system into three independent layers so each could be built, tested and swapped on its own:

  1. Reasoning layer. A local LLM works out the gap between where someone is and where they want to go.
  2. Retrieval layer. Vector search maps each gap to real courses in the catalog.
  3. Analytics layer. Pandas pipelines turn 1.6 GB of raw job market data into small, fast summary files for the dashboard. Streamlit sits on top as the interface. Statistical queries go to the precomputed analytics files. Natural-language input goes to the LLM.
LayerTools
InterfaceStreamlit, Altair
Reasoningllama-cpp-python running Mistral-7B-Instruct v0.2 (Q4_K_M GGUF)
Retrievalsentence-transformers (all-MiniLM-L6-v2), FAISS-CPU
Analyticspandas, NumPy
EnvironmentPython 3.11 on CachyOS

Running a 7B model on a laptop CPU

A full-precision 7B model needs around 14 GB of memory, which doesn’t leave room for anything else on a 16 GB machine. The fix is quantization.

I used mistral-7b-instruct-v0.2.Q4_K_M.gguf. GGUF is the single-file format llama.cpp loads, with the weights, tokenizer and metadata bundled together. Q4_K_M stores weights at roughly 4 bits using a “medium” k-quant scheme. That brought the model down to about 4.1 GB with reasoning quality that held up fine for this task.

Loading it looks like this:

import multiprocessing
from llama_cpp import Llama
 
llm = Llama(
    model_path="models/mistral-7b-instruct-v0.2.Q4_K_M.gguf",
    n_ctx=2048,                                   # keeps the memory buffer safe
    n_threads=multiprocessing.cpu_count() - 1,    # leave one core for the OS
    verbose=False,
)

Two small settings mattered a lot:

  • cpu_count() - 1 threads. Using every core made the whole machine freeze during generation. Leaving one free kept the UI and OS responsive.
  • A 2,048-token context. My prompts are short, so a bigger context would only waste memory and risk crashes.

Making an LLM return clean data

The analyzer needs the model to return a list of skills, not a friendly paragraph. That’s harder than it sounds, because instruction-tuned models love to chat.

The prompt casts the model as an expert HR career advisor and demands a plain comma-separated list of skill names, with no explanations and no parentheses. I also locked down the generation settings:

  • temperature=0.1 for consistent, near-deterministic output
  • max_tokens=100 to stop runaway answers Even so, the model would sometimes open with “Here are the missing competencies:” or end with “Note: these may vary…”. So every response goes through a cleanup pipeline before anything downstream sees it:
import re
 
def clean_skills(text: str) -> list[str]:
    # 1. Remove conversational preambles
    text = re.sub(r"^.*?(here are|missing (skills|competencies))[^:]*:\s*", "",
                  text, flags=re.IGNORECASE)
 
    # 2. Cut off trailing disclaimers
    lower = text.lower()
    for marker in ("note:", "however", "please", "keep in mind"):
        idx = lower.find(marker)
        if idx != -1:
            text, lower = text[:idx], lower[:idx]
 
    # 3. Strip anything in parentheses
    text = re.sub(r"\(.*?\)", "", text)
 
    # 4. Split, trim, and drop anything too long to be a skill name
    return [s.strip() for s in text.split(",")
            if s.strip() and len(s.strip()) <= 50]

The last step is my favourite. Any item longer than 50 characters is almost certainly a sentence that slipped past the regex, so it’s thrown out. It’s a crude rule, but it works.

Lesson: don’t trust LLM output as data. Constrain it with the prompt, then validate it in code anyway.

Retrieval: from skill gaps to real courses

Once the model has named the gaps, the system needs to recommend courses that actually exist. That’s a retrieval problem, and keyword search is bad at it. “Optimizing cloud infrastructure” and “improving cloud system performance” share almost no words but mean nearly the same thing.

So I embedded the title, description and learning objectives of over 5,000 courses with all-MiniLM-L6-v2, which turns text into 384-dimensional vectors where similar meanings sit close together.

Why such a small model? It has about 22 million parameters, runs fast on a CPU and is accurate enough for short skill phrases. Larger 768- or 1536-dimensional models would cost more memory and time for very little gain here.

Choosing the FAISS index

FAISS offers clever approximate indexes like HNSW and IVF, but they’re built for millions or billions of vectors and need tuning. With about 5,000 courses, I used a flat index (IndexFlatL2). It compares the query against every vector, so it’s 100% exact and still instant at this size.

The index is built once, saved to disk with faiss.write_index, and loaded at startup with faiss.read_index.

Two tricks that made retrieval actually good

1. Query anchoring. My first version searched for the bare skill, like “Python”. It returned beginner scripting tutorials to someone aiming to be a data scientist. The fix was to prepend the target role to every query:

query = f"{target_role}: {skill}"
vector = embedder.encode([query]).astype("float32")
distances, ids = index.search(vector, k=3)

The role pulls the search into the right region of the vector space. This one change improved recommendations more than anything else I tried.

2. A distance cut-off. FAISS always returns something, even when nothing is relevant. I discard any result with a distance above 2.5, so the system shows nothing rather than a wrong course:

matches = [courses[i] for d, i in zip(distances[0], ids[0]) if d <= 2.5]

An empty result is honest. A bad recommendation wastes an employee’s time and erodes trust in the tool.

The data engineering side

The HR dashboard runs on more than 1.6 GB of job market data: job postings with salaries, and a large file mapping each posting to its required skills. Loading that into pandas in one go would crash a 16 GB laptop, so I built the analytics as offline pipelines that produce small, ready-to-plot CSVs.

Sampling a file too big to load

The skills file is read in chunks, sampled, and stopped early:

chunks = []
for i, chunk in enumerate(pd.read_csv(RAW_DATA_DIR / "job_skills.csv",
                                      chunksize=100_000)):
    chunks.append(chunk.sample(frac=0.01, random_state=42))
    if i + 1 >= 50:          # ~5M rows scanned
        break
 
sample = pd.concat(chunks)
sample = sample.sample(n=min(5_000, len(sample)), random_state=42)
sample.to_csv(PROCESSED_DATA_DIR / "sampled_job_skills.csv", index=False)

The fixed random_state means anyone running the pipeline gets exactly the same sample.

Salary benchmarks

From the postings, I kept only the columns I needed (usecols) and only yearly pay. I then estimated annual salary as the median where available, or otherwise the midpoint of the minimum and maximum. Titles were lowercased and trimmed, and any title appearing just once was dropped as noise. That left benchmarks for 3,609 job titles, each with its top three hiring companies.

Role-specific skill demand

This is the metric I’m proudest of. Skills arrive as comma-separated strings, so I split them into lists and used pandas .explode() to give each skill its own row. A posting needing five skills becomes five rows, and demand becomes a simple count.

The important part is that demand is calculated within each job title, not globally. “Kubernetes appears in 3% of all postings” is useless to an L&D team. “Kubernetes appears in 64% of DevOps Engineer postings” is something you can act on. Any skill appearing in under 2% of a title’s postings is dropped, which removes typos and one-off noise.

Finding training gaps

The final pipeline takes the top 50 in-demand skills and counts how many catalog courses cover each one:

Matching coursesStatus
0Critical
1–2Low coverage
3–5Moderate
More than 5Covered

Critical gaps in the highest-demand skills are sorted to the top. That’s the list an L&D head can take straight into a budget conversation.

Matching messy job titles

People don’t type job titles the way datasets store them. Someone enters “Ethical Hacker” while the data says “Cyber Security Specialist”. I wrote a four-tier fallback so a lookup almost always lands somewhere sensible:

  1. Exact match (case-insensitive)
  2. Forward substring: the input sits inside a dataset title (“Analyst” → “Senior Data Analyst”)
  3. Reverse substring: a dataset title sits inside a long, descriptive input
  4. Keyword overlap scoring: tokenize, remove stop words, and pick the title with the most shared words Each tier only runs if the one before it fails, so common cases stay fast.

Keeping the interface fast

A 4 GB model and a pile of CSVs shouldn’t reload every time someone clicks a button. Streamlit’s caching handled this:

  • @st.cache_resource keeps the loaded LLM and the course recommender in memory across sessions, so the model loads from disk once.
  • @st.cache_data caches the six processed analytics files, with try/except so a missing file shows an empty state instead of crashing the app. I also pre-generated the advisor answers. HR leaders tend to ask the same strategic questions, such as “Which skills should we prioritize for L&D investment this quarter?” A standalone script feeds the key metrics and those questions to the model offline (capped at 400 tokens each) and saves the answers to advisor_cache.json. The dashboard reads from that file, so those answers appear instantly instead of waiting on the CPU.

The interface uses a custom Nord colour theme injected through CSS, with Altair handling the charts: a top-20 skills bar chart, donut charts for experience level and remote/hybrid split, and colour-coded gap statuses.

Why CachyOS helped

I built and ran everything on CachyOS, an Arch-based distro that compiles its packages for newer x86-64-v3 instruction sets and ships latency-focused CPU schedulers. For CPU-bound inference, that’s a real advantage. The scheduler also kept the Streamlit UI responsive while the model was generating in the background.

What came next: Llama 3.1 8B

After the project, I swapped Mistral 7B for Llama 3.1 8B Instruct, also in Q4_K_M. It’s slightly larger (around 4.9 GB) but follows formatting instructions more reliably. That means fewer preambles and disclaimers for the cleanup pipeline to catch, and fewer invented skills. Because the reasoning layer is decoupled, the swap was essentially a new model path. Nothing else in the system had to change.

What I learned

  • Local AI is practical now. A quantized 7B model on a five-year-old laptop CPU gave genuinely useful reasoning, with zero API cost and zero data leaving the machine.
  • Keep the LLM’s job narrow. The model only names the gaps. Vectors handle matching, and pandas handles the numbers. Small models get vague when asked to do everything at once.
  • Retrieval quality is mostly design, not model size. Query anchoring and a distance threshold beat any embedding upgrade I considered.
  • Most of the work was data engineering. Chunked reads, sampling, explode-and-count, noise thresholds and caching made up far more of this project than prompting did. That’s a big part of why I’m now moving into data engineering.

What I’d do differently

  • Use DuckDB instead of sampling. Sampling down to 5,000 skill rows kept memory safe, but it threw away a lot of signal. DuckDB can query the full files on disk without loading them into memory.
  • Build an evaluation set early. I judged gap and course quality by reading outputs. A small set of role pairs with expected answers would have made every change measurable.
  • Ask the model for JSON. Structured output with grammar-constrained generation in llama.cpp would replace most of the regex cleanup.
  • Re-index automatically. The course index is static. In production, a scheduled job should re-embed the catalog whenever the LMS changes. This started as an MBA research project, but it changed how I think about AI. The interesting part isn’t the model. It’s the plumbing around it.