Project Lion - Senior Prompt Engineer - Japan (Remote, Part-Time)

<p> We are seeking a <strong>Prompt Engineer</strong> to be responsible for the end-to-end technical migration workflow for transitioning templates to LLM autoraters. The role is required to use client’s internal tools to leverage prompt engineering techniques to maximize model performance. </p> <h3>Responsibilities: </h3> <ul> <li> <p>Utilize Automatic Prompt Generation (APG) tools to create baseline prompts for complex parent-child template clusters. </p> </li> </ul> <ul> <li> <p>Run and supervise Automated Prompt Optimization (APO) tool, review the outputs, and flag when the APO reaches deadlocks or plateaus.   </p> </li> </ul> <ul> <li> <p>Manually draft, test, and refine prompts to navigate complex template architectures, overcome anti-patterns, and handle edge cases where tooling is lacking or broken. Solve edge-case scenarios by designing and refining manual prompts. </p> </li> </ul> <ul> <li> <p>Monitor shadowbot runs to ensure sufficient disagreements (between human and LLM ratings) are registered, generated, and tracked.   </p> </li> </ul> <ul> <li> <p>Run prompt versions against established gold data to continuously measure autorater quality against the human crowd baseline, calculating accuracy metrics such as F1 scores, precision, and recall.   </p> </li> </ul> <ul> <li> <p>Draft technical launch readiness justifications (Launch Certification Documentation) for final. </p> </li> </ul> <h3>Requirement: </h3> <ul> <li> <p><strong>Language Skills</strong><strong>:</strong> Native fluency in Japanese and fluent in English. </p> </li> </ul> <ul> <li> <p><strong>Location:</strong> Must be based in Japan.</p> </li> </ul> <ul> <li> <p><strong>Education</strong><strong>:</strong> Bachelor’s, Master’s, or Doctorate degree in Computer Science, Data Science, Computational Linguistics, Human-Computer Interaction (HCI), Cognitive Science, or a related analytical field.  </p> </li> </ul> <ul> <li> <p><strong>Prompt Engineering &amp; AI Expertise</strong><strong>:</strong> At least 4 years' experience as Prompt Engineer. Proven experience tuning Large Language Models (LLMs) for strict, structured outputs, complex classification tasks, and familiarity with chain-of-thought and few-shot learning.  </p> </li> </ul> <ul> <li> <p><strong>Data Analysis</strong><strong>:</strong> Strong proficiency in identifying error patterns, analyzing model performance, and using SQL or other data analytics tools.  </p> </li> </ul> <ul> <li> <p><strong>Technical Agility</strong><strong>:</strong> Ability to quickly learn and master proprietary tools with minimal supervision.  </p> </li> </ul> <ul> <li> <p><strong>Communication</strong><strong>:</strong> Excellent verbal and written communication skills.  </p> </li> </ul> <h3>Optional / Preferred Skills:  </h3> <ul> <li> <p>Familiarity with enterprise-grade LLM interfaces like the Goose API.  </p> </li> </ul> <ul> <li> <p>Experience in AI model evaluation, data science, computational linguistics, or software engineering.  </p> </li> </ul> <ul> <li> <p>Hands-on experience with Automated Prompt Optimization (APO) systems or tuning workflows.  </p> </li> </ul> <ul> <li> <p>Linguistic expertise, including an understanding of semantics and logic.  </p> </li> </ul> <h3>Compensation</h3> <h3>Additional Information</h3>