Skip to main content
The Qualia hillclimb preview is an optional plugin for your local coding agent. Use it to improve a training recipe, prompt, retrieval pipeline, or algorithm against a fixed evaluation. Qualia runs the experiments in Cloud; your coding agent reviews the results and integrates the candidate into your project.
This preview is distributed as a local plugin bundle. It is separate from the skills bundled inside Qualia. You need Node 22 or newer and a Qualia Cloud account.

Install the preview

Unzip the supplied Qualia hillclimb bundle into a stable folder on your computer. The bundle includes the skill and its helper; you do not need the Qualia CLI.

Claude Code

Launch Claude Code with the plugin folder:
Invoke /qualia:hillclimb with your objective. The plugin includes the Qualia MCP connection. To receive Qualia questions and completed messages while Claude is idle, the Qualia channel must also be enabled when launching Claude. This is currently a Claude Code research-preview feature; Team and Enterprise organizations need administrator enablement. Follow the channel setup in the bundle’s README. Without it, the plugin uses one long event wait. Interactive Claude Code 2.1.212+ can move that wait to a background task and notify the conversation when an update arrives, depending on your background-task settings.

Codex

Copy the bundle’s skills/hillclimb folder into your project’s .agents/skills/hillclimb folder, then register its helper:
Restart Codex and invoke $hillclimb with your objective.

Sign in

Your coding agent checks for an existing local Qualia login. If needed, it runs the bundled helper and opens your browser. Complete sign-in on the same computer as your coding agent. Signing into the website in a separate tab does not by itself connect the helper. After a successful sign-in, the browser returns to your Cloud workspaces. If an old login tab shows “localhost refused to connect,” ask your coding agent to check the Qualia connection. When that check succeeds, you can close the old tab without signing in again. The helper stores your session locally with owner-only permissions. Do not paste credentials into chat. API keys alone do not authenticate this preview’s Cloud research workflow.

Start a research run

For example:
Improve this classifier’s validation accuracy. Keep the evaluator and validation split fixed. Try at most five CPU-only experiments and return the best code and a report of every trial.
Your coding agent and Qualia settle the evaluation, permitted changes, compute limits, and stopping condition. If there is no starting code, Qualia can build a minimal baseline; your coding agent handles application-specific setup. For a small public-dataset demo, you can start with just the dataset and metric:
California housing dataset, minimize MSE.
When you have not supplied limits, the agent states a default scope of five candidate trials, one CPU worker and 20 minutes. Qualia prepares a deterministic evaluation and baseline in Cloud. Larger or expensive runs need an explicit scope. In Claude Code the agent asks the setup decisions as a short multiple-choice survey: the metric and its constraints, how scores and failures are counted, the budget, and which numbers you already trust. If you say you are away, it picks documented defaults instead and lists them in the final summary. Before any experiment runs, Qualia freezes the evaluator and computes the values the objective depends on. Your agent shows a kickoff card that leads with any target that disagrees with your reference numbers and any failed or warning data check, then the contract: objective, evaluation, data, scoring rules, budget and approval policy. A mismatch stops the run until the contract is fixed. Command approvals have two modes, chosen in the setup survey before the run is created. In the recommended mode, which is also the default when you are away, Qualia runs commands in its workspace sandbox without asking. A few commands still need confirmation, such as deleting a home or root directory or writing outside the workspace when no sandbox confines the command. In the other mode every command asks. There, accepting the card’s approval policy lets the helper approve read-only commands and runs of the frozen evaluator. That approval lasts for that run only, and everything else still comes to the agent. The agent posts a clickable link to your Qualia workspace as soon as it is created. Ask “Where’s the Qualia workspace?” at any time to get the same link. The research kickoff identifies your local coding agent and invokes Qualia’s existing /optimize workflow: baseline and experiment tasks, a leaderboard, measured evidence and verification. Your explicit limits take precedence over the workflow’s default size. When source files are needed, your coding agent selects them explicitly. The helper uploads their current contents, including uncommitted edits, and saves a baseline copy locally. Transfers support up to 200 files, 8 MiB per file, and 32 MiB total. Use remote references for larger datasets and model checkpoints. Trial, time, and spend limits are instructions to the research agent. They are not enforced spending caps. Keep a separate held-out evaluation locally when you need independent confirmation of improvements.

Check progress and retrieve results

Qualia acknowledges the run in chat and reports measured milestones. Your local agent shows each milestone as a small leaderboard (best first, the baseline row, new results and a new best marked, warnings underneath) and raises integrity problems, such as an edited evaluator or a marginal pass, as soon as they appear. It summarizes the baseline, meaningful improvements, blockers and final result; empty waits and routine connection recovery stay quiet. Ask your coding agent a progress question or give it a correction; it can steer the running Qualia agent using the same action as Send now in Cloud. This may interrupt the current step. If Qualia has not answered within the short wait, your coding agent continues waiting without guessing from workspace files. When Qualia or one of its optimization workers asks a question, your local agent answers from the context it already has or brings the decision to you. Command approvals follow the same route and apply only once, within your authorized scope. Routine approvals are handled quietly; a Cloud execution prompt does not by itself require a new decision from you. The agent checks unfamiliar evaluator behavior when needed and brings scope changes or unresolved permissions to you. Completed Qualia messages can arrive before all research is finished. Enabled Claude channels push these events into an open conversation. Without channels, Claude runs default to a single ten-minute wait; other hosts use 45-second waits. If Claude moves the wait to a background task, the agent waits for its completion notification without additional progress checks. The helper reconciles reconnects and checks status while disconnected. Closing the local conversation stops event delivery but not Cloud work. After restarting the local agent, ask it to resume monitoring the saved run. At the end Qualia reports two checks: a reproduction of the winning method and a ship check of the artifact exactly as shipped. Your agent reruns both locally on the exact inputs you uploaded, after reviewing the commands, and writes a summary with the verified numbers. A run can reproduce and still fail its ship check; the agent then reports that the method works but the artifact is not ready to use. The coding agent reads the experiment report, downloads selected candidate files into a new staging folder, and compares them against both the uploaded baseline and your current project. Your working files are not overwritten by download. It can then rerun the evaluation and integrate the result within your requested scope. A run may conclude that none of its candidates improved on the baseline. To stop, ask your coding agent to stop the Qualia run. The stop request halts the research chat; any independently launched training jobs must also be checked.