How to create a coding test

What you'll need

• A SkillCort workspace (or a demo)
• A real example of the code your team writes — a bug, a small feature, a refactor
• A senior engineer to define the task and the rubric
• A decision on which languages you will accept

Step 1: Choose a realistic task over an algorithm puzzle

Look at your last month of pull requests and pick a task shaped like the ones that recur: fix a bug given a failing behavior, extend a small function to handle a new case, refactor a messy block, or write a data transformation. These predict day-to-day performance far better than inverting a binary tree.

Size it to 30–45 minutes of focused work for a competent hire. A puzzle rewards whoever has rehearsed it; a realistic small task rewards whoever can read existing intent, make a sound change, and keep the code understandable — which is what you are hiring for.

Strip framework-specific and company-specific knowledge unless the role genuinely requires it, and provide any context the task needs inside the brief itself. A strong engineer who has never touched your stack should still be able to show you sound judgment. Test the skill, not familiarity with your codebase's trivia.

• Recurs in your real codebase, not in interview prep books
• Has more than one reasonable solution
• Fits 30–45 minutes without heroics
• Readable by an evaluator in under ten minutes

Step 2: Create the code item in the items library

Make a folder for the role — Backend Engineer, for example — with tags for skill area and difficulty, then create an item using the code item type. Candidates write and edit code directly in the response area, and evaluators later review it in place.

Seed the item with starter code rather than a blank editor. A short function with a described bug, or a stub with a clear contract, anchors every candidate to the same starting point and makes their changes directly comparable. Real work almost never starts from an empty file, and neither should the test.

Step 3: Write the brief beside the editor

Use the side-by-side stimulus layout: the brief, requirements, and example inputs and outputs on one side, the code editor beside it. The candidate never scrolls away from the spec to write the solution — the same shape as working from a ticket in an IDE.

Write the brief the way a good ticket reads: current behavior, expected behavior, constraints, and two or three concrete examples. State explicitly what you care about — "working code first; we also read for clarity" — so candidates optimize for what you will actually score. Ambiguity in the brief becomes noise in the results.

Step 4: Build a rubric that scores more than correctness

Correct output is necessary, not sufficient. Build a level-based matrix rubric with three or four criteria — typically correctness, readability, and approach, optionally edge-case handling — each with anchored descriptions of what strong, acceptable, and weak code looks like. "Names reveal intent, no dead code" is scoreable; "clean code" is not.

Weight correctness heaviest, but keep the other criteria real: a barely working solution nobody can maintain is not a strong hire signal. When you attach the rubric it freezes into a snapshot, so every candidate in the run is reviewed against the identical standard even if the bank version evolves later.

Step 5: Assemble and timebox in the builder

Create the assessment as sections, pages, and blocks and import the code item through the picker. Many teams add a short first section of quick judgment questions — reading a diff, spotting the bug in a snippet, quick-pasted as multiple choice — and put the coding task in its own section.

Set the total time limit with slack for reading and thinking, and use a per-page limit on the coding page so the task cannot swallow the whole attempt. Weight the coding section heaviest (for example ×3) so role-fit on the Decision Board reflects the work, not the warm-up. Write instructions that name the allowed languages and what happens if time runs out.

Step 6: Preview, clear the checklist, and publish

Use Preview as candidate and attempt the task yourself, or better, have an engineer who did not write it attempt it cold. If they finish in ten minutes it is too easy; if they cannot finish in the limit, loosen the scope or the clock. Fix the brief wherever they hesitated.

Then satisfy the publish checklist — it blocks Publish until scoring, timing, and instructions are complete. A coding test with a missing rubric or an untested time limit should never reach a candidate, and the checklist makes sure it cannot.

Step 7: Set integrity controls proportional to the stakes

Create the delivery with a name and time window, then choose security toggles deliberately. For a standard hiring screen, copy/paste blocking and fullscreen lockdown with a sensible violation limit are usually enough. Webcam recording, screen recording, second-screen detection, and ID checks exist for genuinely high-stakes runs — certification, final rounds — not as defaults.

Everything you enable is disclosed at the consent and preflight gate before the candidate starts. Integrity signals are flags for a human to review in context, never grounds for automatic rejection — a fullscreen exit can be a notification, not fraud. Proportionality keeps strong candidates from walking away from a surveillance dragnet.

Step 8: Review code with the rubric, with AI as support

Evaluators review each submission against the shared rubric, clicking the level card that matches the code on each criterion — correctness, readability, approach. Putting two reviewers on borderline or high-stakes cases keeps the standard honest, and because everyone scores against the same frozen rubric snapshot, disagreements surface as specific criterion gaps rather than vague gut-feel differences.

AI can draft a summary of the submission and suggest criterion-level scores — a genuinely useful first pass on code, with the model and version logged for the record. But a named engineer confirms or overrides every score. AI is decision support here, never the decision-maker: the accountable judgment about a candidate's code is always human.