Data & method

Everything SkillTide shows comes from the sources and formulas on this page. Predictions are estimates for exploring options, not forecasts to bet a career on.

Data updates

Refresh not yet verified

Last validated refresh2026-10-21 17:26 UTC

Current dataset build2026-10-21 17:26 UTC

Substantive data changed in this build. Build dates are not source publication dates. Fixed AI releases are curated; hiring signals and the live course catalogue are refreshed by the weekly pipeline.

61 inputs downloaded · 0 reused · 0 download dates not recorded.

How it fits together

Dataset generated 2026-10-21 by pipeline/build_dataset.py; the web app only reads that file.

  1. 01

    Collect

    Download dated public data: O*NET 31.0, Anthropic Economic Index releases, Indeed postings, Hacker News hiring threads, UN SDG targets, course catalogues.

  2. 02

    Signal

    Turn them into per-task AI exposure, per-role hiring demand and per-skill momentum — one set for each dated snapshot.

  3. 03

    Score

    The engine projects each task under a scenario, rolls tasks up to roles, then matches roles to your skills and goals.

  4. 04

    Plan

    Address priority gaps with real courses whose hours and cost we could check, ending with a portfolio project.

1. AI exposure of each task

Following Anthropic's idea of combining what models could do with what people actually use them for

ceiling c   = β from Eloundou et al. (1: an LLM alone could halve the task's time, 0.5: with extra tools, 0: no)
today   e₀  = c × usage intensity      usage intensity = Claude share of the task ÷ 90th-percentile task share,
                                      at least 0.2 for any task the Anthropic Economic Index publishes
in h years  e(h) = c − (c − e₀) × (1 − r)^h        r = the scenario's yearly adoption rate

Each task also gets an automation share: of the conversations about it, how many hand the work over (directive or feedback-loop use) rather than work alongside a person (iteration, learning, validation). That splits exposure into work AI is likely to automate and work it is likely to assist.

2. Role outlook

Tasks weighted by O*NET importance, projected 3 years ahead

outlook = 50 + 20 × demand + 40 × assisted share − 55 × automated share + 5 if on MyCOL     (0–100)
demand  = (sector postings year-on-year − all postings year-on-year) ÷ 15%, capped at ±1
MyCOL   = the role is on Malaysia's Critical Occupations List (TalentCorp), i.e. hard to fill locally

Labels compare each role with all 30 roles SkillTide tracks: at least 0.6 standard deviations above the average is rising, 0.6 below is under pressure, and a role where AI touches at least 45% of the work is transforming. Demand is measured against all postings so a hiring slump that hits every sector does not make every role look bad.

Hiring data comes by Indeed sector, not by job title, so roles that share a sector share a demand value. In the latest snapshot (2026-10-16), 12 roles drawing on 3 sectors are at the cap; between them, outlooks differ only through the AI signals and the MyCOL bonus.

3. Skill trends, matching and pathways

skill score = 50 + 25 × momentum + 30 × role signal
  momentum    = change in the share of Hacker News hiring posts mentioning the skill (last 6 months vs the 6 before; ±50% → ±1)
                AI skills average this with the change in the share of Indeed postings that mention AI
  role signal = how far the outlook of the roles needing the skill sits from the average role (±12 points → ±1)

match = (w_skills × skills fit + w_outlook × outlook + w_sdg × SDG fit) ÷ (w_skills + w_outlook + w_sdg)
  skills fit = importance-weighted share of the role's skills you have at the expected level
               (another programming language you know counts as 70% of the way)
  SDG fit    = share of your chosen goals the role contributes to (switched off if you pick none)

Role requirements come from O*NET: technologies O*NET flags as in demand for that occupation, rated skills and knowledge, and skills named in its task statements. Each requirement keeps that evidence. The pathway addresses the highest-priority technical gaps first, choosing courses that cover the most priority per hour of study within your budget, schedules them by your weekly hours, and ends with a ~20-hour portfolio project built on a real task from the role that AI assists rather than replaces.

4. Scenarios

Two futures instead of one prediction

  • Moderate AI r = 0.15/yr

    Each year AI use closes 15% of the gap between how much it is used on a task today and what current models could do, and the split between automating and assisting stays as observed today.

    The 15% yearly pace is SkillTide's assumption, not a measured rate.

  • Fast AI r = 0.35/yr

    AI use closes 35% of the remaining gap each year, and the shift from assisting to automating keeps going at the pace measured in the Anthropic Economic Index (43% of use was automation in Feb–Mar 2025, 49% in May 2026).

    The 35% yearly pace is SkillTide's assumption; the automation trend is measured.

Measured: automation share of Claude use (usage-weighted)

  • Feb–Mar 202543%
  • 4–11 Aug 202552%
  • 13–20 Nov 202547%
  • 5–12 Feb 202646%
  • May 202649%

5. Sources

Every chip in the app links back to one of these

SourceUsed forLicencePublished / data as ofLatest recorded download
O*NET 31.0 Database

U.S. Department of Labor, Employment and Training Administration

Roles, their tasks and task importance, technologies in demand, rated skills and knowledge, new tasksCC BY 4.02026-082026-10-21 17:23 UTC

10/10 input dates recorded

GPTs are GPTs task exposure labels (β)

Eloundou, Manning, Mishkin & Rock — OpenAI / OpenResearch / UPenn

Theoretical exposure ceiling (β) for each taskMIT20242026-10-21 17:23 UTC

1/1 input dates recorded

Anthropic Economic Index

Anthropic

How much Claude is used for each task, how (automating vs assisting), and Malaysia's usage indexCC BY (data)2026-06-262026-10-21 17:23 UTC

7/7 input dates recorded

Anthropic — Labor market impacts (observed exposure method)

Anthropic (Massenkoff & McCrory)

The idea of combining theoretical exposure with observed useCited2026-03-05Not recorded

Curated reference

Anthropic — What work can robots do? (robot exposure index)

Anthropic

Robot exposure index shown on role cards (context only)CC BY (data)2026-09-302026-10-21 17:23 UTC

1/1 input dates recorded

Indeed Hiring Lab — AI Tracker (share of postings mentioning AI)

Indeed Hiring Lab

Share of postings that mention AI, for AI skills' momentumCC BY 4.02026-09-302026-10-21 17:23 UTC

1/1 input dates recorded

Indeed Hiring Lab — Job Postings Index by sector

Indeed Hiring Lab

Hiring demand: postings by sector, relative to all postingsCC BY 4.02026-10-162026-10-21 17:23 UTC

6/6 input dates recorded

Hacker News “Who is hiring?” threads (via Algolia HN API)

Y Combinator / Algolia

Mentions of each skill in monthly hiring threads, for skill momentumCounts only; text not stored2026-10-012026-10-21 17:24 UTC

33/33 input dates recorded

UN SDG Global Database — goals and targets

United Nations Statistics Division

Goal and target wordingCitedNot recorded2026-10-21 17:26 UTC

1/1 input dates recorded

Microsoft Learn Catalog API

Microsoft

Free learning paths with their durationsMicrosoft APIs Terms of Use (links and metadata only)Not recorded2026-10-21 17:26 UTC

1/1 input dates recorded

SkillTide course checks

SkillTide team

Other free and low-cost courses, each with the page its hours and cost came fromCC BY 4.02026-10-21Not recorded

Curated reference

MyMahir Malaysia Critical Occupations List (MyCOL) 2024/2025 - 8th edition, 66 critical occupations across 18 sectors

TalentCorp Malaysia

Malaysian demand: roles on the Critical Occupations List get a small outlook bonusCited2025Not recorded

Curated reference

New-occupation lists and studies (MOHRSS China, TalentCorp, WEF)

Various; each item links to its own source

Newly recognised roles shown on the radar, each with its own sourceCited2026-10-21Not recorded

Curated reference

This page includes information from the O*NET 31.0 Database by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA). Used under the CC BY 4.0 license. O*NET® is a trademark of USDOL/ETA. SkillTide has modified all or some of this information. USDOL/ETA has not approved, endorsed, or tested these modifications.

6. Limits we know about

  • O*NET describes US occupations. Malaysian roles of the same name are similar but not identical.
  • The Anthropic Economic Index covers people who use Claude, not all AI use. Releases differ in sample and method, so part of the change between snapshots is measurement, not the world.
  • Indeed's postings data covers the US, UK and Australia, not Malaysia; Malaysia appears through Claude usage and the context figures we cite.
  • Hacker News hiring threads lean towards US software start-ups, so they say little about electronics or automation skills (those show “too few to tell”).
  • The adoption rates in the two scenarios are assumptions. The labels are relative: “under pressure” means a weaker outlook than most roles here, not that a job will vanish.
  • Course lists are not exhaustive: SkillTide only lists courses whose length and cost it could check.

7. Responsible AI

  • SkillTide supports your decisions; it does not make them, and it is not a hiring tool.
  • Scoring never uses name, age, gender, ethnicity or background. Every match is re-checked against profile variants with different names and graduation years, and must come out identical.
  • Your profile stays in your browser. Sample students are synthetic; please never paste someone else's CV.
  • CV reading and the guide's answers are rule-based. No language model runs in the product. Recommendations use published data, explicit formulas and stated scenario assumptions.
  • Historical views replay past signals with the current role catalogue and scenario assumptions; they are not a historical forecasting backtest. Learning plans address skill gaps, but completing a course does not establish mastery.