Data & method
Everything SkillTide shows comes from the sources and formulas on this page. Predictions are estimates for exploring options, not forecasts to bet a career on.
Data updates
Refresh not yet verifiedLast validated refresh2026-10-21 17:26 UTC
Current dataset build2026-10-21 17:26 UTC
Substantive data changed in this build. Build dates are not source publication dates. Fixed AI releases are curated; hiring signals and the live course catalogue are refreshed by the weekly pipeline.
61 inputs downloaded · 0 reused · 0 download dates not recorded.
How it fits together
Dataset generated 2026-10-21 by pipeline/build_dataset.py; the web app only reads that file.
01
Collect
Download dated public data: O*NET 31.0, Anthropic Economic Index releases, Indeed postings, Hacker News hiring threads, UN SDG targets, course catalogues.
02
Signal
Turn them into per-task AI exposure, per-role hiring demand and per-skill momentum — one set for each dated snapshot.
03
Score
The engine projects each task under a scenario, rolls tasks up to roles, then matches roles to your skills and goals.
04
Plan
Address priority gaps with real courses whose hours and cost we could check, ending with a portfolio project.
1. AI exposure of each task
Following Anthropic's idea of combining what models could do with what people actually use them for
ceiling c = β from Eloundou et al. (1: an LLM alone could halve the task's time, 0.5: with extra tools, 0: no)
today e₀ = c × usage intensity usage intensity = Claude share of the task ÷ 90th-percentile task share,
at least 0.2 for any task the Anthropic Economic Index publishes
in h years e(h) = c − (c − e₀) × (1 − r)^h r = the scenario's yearly adoption rateEach task also gets an automation share: of the conversations about it, how many hand the work over (directive or feedback-loop use) rather than work alongside a person (iteration, learning, validation). That splits exposure into work AI is likely to automate and work it is likely to assist.
2. Role outlook
Tasks weighted by O*NET importance, projected 3 years ahead
outlook = 50 + 20 × demand + 40 × assisted share − 55 × automated share + 5 if on MyCOL (0–100) demand = (sector postings year-on-year − all postings year-on-year) ÷ 15%, capped at ±1 MyCOL = the role is on Malaysia's Critical Occupations List (TalentCorp), i.e. hard to fill locally
Labels compare each role with all 30 roles SkillTide tracks: at least 0.6 standard deviations above the average is rising, 0.6 below is under pressure, and a role where AI touches at least 45% of the work is transforming. Demand is measured against all postings so a hiring slump that hits every sector does not make every role look bad.
Hiring data comes by Indeed sector, not by job title, so roles that share a sector share a demand value. In the latest snapshot (2026-10-16), 12 roles drawing on 3 sectors are at the cap; between them, outlooks differ only through the AI signals and the MyCOL bonus.
3. Skill trends, matching and pathways
skill score = 50 + 25 × momentum + 30 × role signal
momentum = change in the share of Hacker News hiring posts mentioning the skill (last 6 months vs the 6 before; ±50% → ±1)
AI skills average this with the change in the share of Indeed postings that mention AI
role signal = how far the outlook of the roles needing the skill sits from the average role (±12 points → ±1)
match = (w_skills × skills fit + w_outlook × outlook + w_sdg × SDG fit) ÷ (w_skills + w_outlook + w_sdg)
skills fit = importance-weighted share of the role's skills you have at the expected level
(another programming language you know counts as 70% of the way)
SDG fit = share of your chosen goals the role contributes to (switched off if you pick none)Role requirements come from O*NET: technologies O*NET flags as in demand for that occupation, rated skills and knowledge, and skills named in its task statements. Each requirement keeps that evidence. The pathway addresses the highest-priority technical gaps first, choosing courses that cover the most priority per hour of study within your budget, schedules them by your weekly hours, and ends with a ~20-hour portfolio project built on a real task from the role that AI assists rather than replaces.
4. Scenarios
Two futures instead of one prediction
Moderate AI r = 0.15/yr
Each year AI use closes 15% of the gap between how much it is used on a task today and what current models could do, and the split between automating and assisting stays as observed today.
The 15% yearly pace is SkillTide's assumption, not a measured rate.
Fast AI r = 0.35/yr
AI use closes 35% of the remaining gap each year, and the shift from assisting to automating keeps going at the pace measured in the Anthropic Economic Index (43% of use was automation in Feb–Mar 2025, 49% in May 2026).
The 35% yearly pace is SkillTide's assumption; the automation trend is measured.
Measured: automation share of Claude use (usage-weighted)
- Feb–Mar 202543%
- 4–11 Aug 202552%
- 13–20 Nov 202547%
- 5–12 Feb 202646%
- May 202649%
5. Sources
Every chip in the app links back to one of these
| Source | Used for | Licence | Published / data as of | Latest recorded download |
|---|---|---|---|---|
| O*NET 31.0 Database U.S. Department of Labor, Employment and Training Administration | Roles, their tasks and task importance, technologies in demand, rated skills and knowledge, new tasks | CC BY 4.0 | 2026-08 | 2026-10-21 17:23 UTC 10/10 input dates recorded |
| GPTs are GPTs task exposure labels (β) Eloundou, Manning, Mishkin & Rock — OpenAI / OpenResearch / UPenn | Theoretical exposure ceiling (β) for each task | MIT | 2024 | 2026-10-21 17:23 UTC 1/1 input dates recorded |
| Anthropic Economic Index Anthropic | How much Claude is used for each task, how (automating vs assisting), and Malaysia's usage index | CC BY (data) | 2026-06-26 | 2026-10-21 17:23 UTC 7/7 input dates recorded |
| Anthropic — Labor market impacts (observed exposure method) Anthropic (Massenkoff & McCrory) | The idea of combining theoretical exposure with observed use | Cited | 2026-03-05 | Not recorded Curated reference |
| Anthropic — What work can robots do? (robot exposure index) Anthropic | Robot exposure index shown on role cards (context only) | CC BY (data) | 2026-09-30 | 2026-10-21 17:23 UTC 1/1 input dates recorded |
| Indeed Hiring Lab — AI Tracker (share of postings mentioning AI) Indeed Hiring Lab | Share of postings that mention AI, for AI skills' momentum | CC BY 4.0 | 2026-09-30 | 2026-10-21 17:23 UTC 1/1 input dates recorded |
| Indeed Hiring Lab — Job Postings Index by sector Indeed Hiring Lab | Hiring demand: postings by sector, relative to all postings | CC BY 4.0 | 2026-10-16 | 2026-10-21 17:23 UTC 6/6 input dates recorded |
| Hacker News “Who is hiring?” threads (via Algolia HN API) Y Combinator / Algolia | Mentions of each skill in monthly hiring threads, for skill momentum | Counts only; text not stored | 2026-10-01 | 2026-10-21 17:24 UTC 33/33 input dates recorded |
| UN SDG Global Database — goals and targets United Nations Statistics Division | Goal and target wording | Cited | Not recorded | 2026-10-21 17:26 UTC 1/1 input dates recorded |
| Microsoft Learn Catalog API Microsoft | Free learning paths with their durations | Microsoft APIs Terms of Use (links and metadata only) | Not recorded | 2026-10-21 17:26 UTC 1/1 input dates recorded |
| SkillTide course checks SkillTide team | Other free and low-cost courses, each with the page its hours and cost came from | CC BY 4.0 | 2026-10-21 | Not recorded Curated reference |
| MyMahir Malaysia Critical Occupations List (MyCOL) 2024/2025 - 8th edition, 66 critical occupations across 18 sectors TalentCorp Malaysia | Malaysian demand: roles on the Critical Occupations List get a small outlook bonus | Cited | 2025 | Not recorded Curated reference |
| New-occupation lists and studies (MOHRSS China, TalentCorp, WEF) Various; each item links to its own source | Newly recognised roles shown on the radar, each with its own source | Cited | 2026-10-21 | Not recorded Curated reference |
This page includes information from the O*NET 31.0 Database by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA). Used under the CC BY 4.0 license. O*NET® is a trademark of USDOL/ETA. SkillTide has modified all or some of this information. USDOL/ETA has not approved, endorsed, or tested these modifications.
6. Limits we know about
- O*NET describes US occupations. Malaysian roles of the same name are similar but not identical.
- The Anthropic Economic Index covers people who use Claude, not all AI use. Releases differ in sample and method, so part of the change between snapshots is measurement, not the world.
- Indeed's postings data covers the US, UK and Australia, not Malaysia; Malaysia appears through Claude usage and the context figures we cite.
- Hacker News hiring threads lean towards US software start-ups, so they say little about electronics or automation skills (those show “too few to tell”).
- The adoption rates in the two scenarios are assumptions. The labels are relative: “under pressure” means a weaker outlook than most roles here, not that a job will vanish.
- Course lists are not exhaustive: SkillTide only lists courses whose length and cost it could check.
7. Responsible AI
- SkillTide supports your decisions; it does not make them, and it is not a hiring tool.
- Scoring never uses name, age, gender, ethnicity or background. Every match is re-checked against profile variants with different names and graduation years, and must come out identical.
- Your profile stays in your browser. Sample students are synthetic; please never paste someone else's CV.
- CV reading and the guide's answers are rule-based. No language model runs in the product. Recommendations use published data, explicit formulas and stated scenario assumptions.
- Historical views replay past signals with the current role catalogue and scenario assumptions; they are not a historical forecasting backtest. Learning plans address skill gaps, but completing a course does not establish mastery.