lee@applied-science:~/cv

~/lee $ whoami

Lee Penkman

Lee Penkman Text Generator logo

Machine learning engineer, VFX/dev tools builder, product founder working across generative AI, graphics, infrastructure, and human-ai interaction I love automation, creativity, outside the box and radically simple, cost effective systems yet leaning into complexity and the bitter lesson when practical :D

leepenkman@gmail.com lee2.app.nz github.com/lee101 AI / ML VFX / Graphics Systems / Product Algorithms / Trading
Worked at
2025 - Present
Independent product work

Founder / Builder - Applied AI Product Ecosystem

I build and run applied AI products across coding agents, model gateways, speech tooling, local memory, media generation, 3D tools, forecasting, and model acceleration.

Technologies: Go, Python, CUDA, Triton, PyTorch, React, FastAPI, Postgres, CuteDSL, Zig, Rust, C++,, ONNX, local GPU serving.

  • Codex Infinity, OpenPaths, CuteDSL, GPUBrain, DictatorFlow, Text Generator, Netwrck, Papers.app.nz, SimplexGen, Helix, and related product systems.
  • Production details I care about: coding automation, AI scaling, observability, provider routing, CI repair loops, deployments, and performance.
Aug 2023 - Aug 2025
Video AI / product ML

Microsoft - Senior Machine Learning Engineer

I worked on Microsoft Clipchamp, a video creation and editing product, collaborating with research teams on AI features that shipped into real creative workflows.

Technologies: PyTorch, Hugging Face, ffmpeg, NLP, deep learning, video processing, production ML.

  • AI video creation.
  • AI video editing.
  • Background removal.
  • AI VFX and media-generation workflows.
Aug 2021 - Jul 2022
Founding ML engineer

Compose.ai YC W21 - Founding Machine Learning Engineer

I was the first ML engineer at Compose.ai, improving the writing assistant, the data process, and the model iteration loop.

Technologies: PyTorch, Hugging Face, React, NLP, Weights & Biases, AWS, spaCy.

  • AI autocomplete.
  • AI writing assistant features for rewriting and rephrasing text.
  • Spell checking, dataset curation, and bias reduction.
  • Virtual assistants, chatbots, model auto-training pipelines, and LLM inference acceleration.
May 2019 - Aug 2021
Image ML / security

Canva - Image Processing / Machine Learning Engineer

I delivered image processing and machine learning capabilities for Canva photo editing, after earlier infrastructure and security engineering work.

Technologies: Terraform, PyTorch, AWS, Docker, Python, GLSL/OpenGL, CI/CD systems.

  • Machine-learning background removal and object detection.
  • Dataset collection and curation pipelines for training ML algorithms.
  • Phishing detection, infrastructure security, penetration testing, and incident response/on-call.
May 2018 - May 2019
Tech lead / data

Fairfax Media - Tech Lead

I led full-stack product, testing, analytics, data engineering, and warehousing work for high-traffic news products.

Technologies: Terraform, Django, AWS, Docker, Node.js, React, Google Ad Manager, Tag Manager, Optimize, Google Cloud.

  • Rolled out paywalls to high-traffic news sites.
  • Scaled and developed high-traffic publishing systems.
  • Built end-to-end analytics pipelines, reports, and video/ad optimization workflows.
Sep 2015 - May 2018
VFX R&D / production tools

Weta Digital - Motion R&D Technical Director

I supported animation, motion editing, crowd simulation, mocap, and virtual artists with production tools, data systems, and reliability engineering.

Technologies: Python, Node.js, React, Ansible, Docker, ELK, Grafana, Redis, RabbitMQ, Postgres, MySQL, Maya, Nuke.

  • Data engineering, syncing, queueing, and reliability engineering for warehousing projects.
  • Production tracking, product-management tools, analytics, batch/realtime warehousing, and report generation.
  • Image-processing pipelines, camera warping, rendering workflows, and artist workflow improvements.
  • Versioning, tracking, publishing, and diffing for 3D artwork, cameras, meshes, textures, shaders, skeletal motion, and time-series data.
  • Distributed 3D data pipelines, format conversion, and performance optimization.
Aug 2014 - Sep 2015
Registry data / visualization

.nz Registry Services - Developer

I worked on data products for the .nz namespace: registry statistics, broadband mapping, topology visualization, and student mentoring.

Technologies: AWS, Python, Puppet, Ruby/Rails, D3, Hive, HBase, Hadoop, Scala/Spark, Postgres, Elasticsearch, Docker, Elastic Beanstalk.

  • Registrar stats dashboard.
  • BroadbandMap.nz.
  • Internet topology visualization in D3.
  • Mentoring student projects in Hive, HBase, and Hadoop.
Dec 2013 - Jul 2014
Gov services / testing

Infoxchange - Developer

I worked remotely in an agile team on government service discovery products, and introduced Jasmine-based behaviour/test-driven development.

Technologies: Python, Puppet, Django, Docker, Bash, Less, Backbone, Bootstrap, PostgreSQL.

  • Human Services Network for NSW Government.
  • Service Seeker, a map-style service discovery product.
  • Testing discipline that made regressions visible and easier to control.
Apr 2013 - Dec 2013
Marketplace engineering

99designs - Developer

I worked on responsive product engineering, cloud systems, distributed workflows, and performance-sensitive marketplace features.

Technologies: PHP, Go, JavaScript/jQuery, AWS, MySQL, MongoDB, SQS.

  • Design annotations for commenting on specific parts of an image.
  • Campaign Monitor to Mailchimp migration with no downtime using SQS.
  • Data migration to a new SSO service.
Nov 2012 - Apr 2013
BI / developer productivity

Kiwiplan - Developer

I worked on special projects covering bug fixing, developer productivity, test automation, data warehousing, and business intelligence.

Technologies: Pentaho BI, Java, ExtJS, MySQL, MSSQL, Jenkins, ETL workflows.

  • Data warehousing and business intelligence solutions.
  • Migration of distributed/service-oriented systems to Spring Remoting and service discovery.
Mar 2012 - Nov 2012
Teaching

University of Auckland - Teaching Assistant

I helped teach Java, Python, and general computing. I like teaching because it forces clear explanations of how computers actually work.

Technologies: Java, Python, practical computing, debugging, teaching labs.

  • Supported students across practical programming, debugging, and core computing concepts.
Nov 2011 - Feb 2012
Internship

Kiwiplan - Developer Intern

I worked through the full software lifecycle: requirements, mockups, client/server modelling, schema design, development, test automation, QA, and a final presentation.

Technologies: Java, ExtJS, MySQL, Maven, SVN, Tomcat, Hibernate, Spring, Balsamiq, Jenkins.

  • Continuous Automated Scheduling Environment, a rule-based scheduling system.
Jan 2010 - Jul 2010
Early web engineering

SGB Services - Web Developer

I was a web developer apprentice at a small startup importing goods and building ecommerce websites to sell them.

Learned: LAMP, HTML, JavaScript, CSS, AdWords, Analytics, AdSense, SEO/SEM, marketing.

  • Designed, built, and marketed ecommerce websites from scratch.
  • Developed a flight-time calculator.

Big Data

I love data modelling, parallel ETL, distributed processing, warehousing, column stores, star schemas, OLAP, and systems that make decisions smarter.

Data Analytics

I love making data understandable with Plotly, D3, SVG, Grafana, Tableau, Google Data Studio, Leaflet, and spatial analysis.

Machine Learning

I love AI/human interaction, interpretability, fairness, distillation, augmentation, PyTorch, TensorFlow, and healthy data/product feedback loops.

Acceleration

I love CUDA, Triton, fused kernels, model serving, compression, quantization, search indexes, embeddings, and scaling compute.

Science

I love tools that accelerate science: search, biology, longevity, deep learning systems, and better ways to connect research to code.

Graphics

I love differentiable rendering, distance fields, differential geometry, procedural art, photogrammetry, and graphics algorithms.

The Web

I love hacking the web, open source, reusable service architectures, mash-ups, search, and the web as the largest knowledge graph.

Games

I love playing and making games, 3D tools, Maya, Blender, Unreal, Unity, procedural systems, and game AI.

VFX

I love pixels, Nuke, After Effects, Houdini, motion capture, video, simulation, compositing, and production tools for artists.

Security

I love cryptography, network protocols, OWASP, pen testing, incident response, CVEs, low-level programming, and postmortems.

Automation

I love not doing things twice: CI/CD, DevOps, Docker, infrastructure as code, repeatable environments, and automated checks.

UX & Testing

I love simple apps, fast prototypes, critical-feature focus, living documentation, browser tests, unit tests, and confidence to change code.

Model gateway

OpenPaths

OpenAI-compatible gateway across chat, image, video, music, speech, transcription, and embeddings, with provider routing and reasoning controls.

Acceleration

CuteDSL

Custom CUDA/Triton kernels, fused model pipelines, Chronos-2 acceleration, and an API surface for fast AI inference and training.

AI art generation

eBank.nz Art Generator

AI art generator and editor with HD diffusion outputs, prompt discovery, style workflows, and subscription-backed generation capacity.

Unified AI API

Text Generator

Text, speech, vision, code generation, prompt tooling, OCR/caption enrichment, and self-hostable model-serving infrastructure.

Speech control

DictatorFlow

Desktop and browser dictation: hotkey recording, local GPU Parakeet ONNX transcription, cloud fallback, command mode, and transcript history.

3D tools

SimplexGen

3D generation and editing app with a Go server, Vite frontend, local asset indexing, mesh tools, generation queues, billing, and R2 storage.

Game

Ethereal Storm

Go/Ebiten twin-stick bullet hell with co-op, procedural terrain, wave survival, bosses, spell combinations, progression, and dynamic audio.

Hiring / marketplace

Hires.nz

Recruitment and product experiments in the same workspace as SimplexGen, with deployed web assets and Go-backed services.

Voice/product comms

Ring.nz

Voice chat with frontier AI friends.

AI memory

GPUBrain

Hybrid semantic and keyword memory store for agents, CAGRA CUDA vector index, static embedding models, quantization, MCP tools, knowledge graph links, and CLI workflows.

Search

Papers

Search engine for papers, code, methods, datasets, evals, semantic search, GPU acceleration (CAGRA), crawler jobs, cuda kernel optimization and quantization.

Forecasting

Stock Prediction

Auto-research and RL trading experiments over Alpaca/Binance, selector evaluation, Chronos-style forecasting, and realistic portfolio probes.

AI data analyst

Helix

AI data scientist product for interpreting datasets, generating insights, and automating reporting with model-backed analysis workflows.

Online learning & RL

ExperimentFlow

Online learning and RL platform — neural contextual bandits, policy optimization, and continual personalization that keeps improving in production.

Stable Diffusion Server

Production-ready image generation server with Flux/SDXL, ControlNet, inpainting, R2/GCS storage, FastAPI, and Gradio tooling.

CuteDSL

AI kernel and model acceleration work spanning Triton, CUDA, model compilation, and equivalence-focused benchmarking.

Text Generator

Self-hostable API-compatible llm, speech, vision, OCR, prompt, and model-routing stack.

GPUBrain Search Engine

Agent memory with semantic search, CAGRA CUDA vector index, static embedding models, quantization, RRF fusion, MCP server, REST API, and CLI.

Stock Prediction

RL and forecasting research code for market prediction, simulations, selectors, backtests, and agentic training runs.

Writing

How.nz

Technical writing archive since 2015: programming, performance, networking, algorithms, graphics, and game-system design.

Crypto forecasting

BitBank.nz

Live cryptocurrency prediction dashboard — time-series transformers (Chronos/TimesFM), RL trading policies, PufferLib agents, custom CUDA & C++ kernels, Triton.

AI stack analysis

StackHack

AI tech-stack analysis tool for breaking down websites, libraries, frameworks, and implementation choices with LLMs.

AI web simulator

WebSim

AI website simulator for generating and iterating web pages with language models, Python services, and browser tooling.

Education

Reading Time

Kids reading helper built with App Engine, Python, and Reveal.js for simple practice flows.

Game

Big Multiplayer Chess

Many-player chess variant with large-board move generation, game heuristics, AI evaluation, and mobile constraints.

Web tool

WebFiddle

Web editing and sharing tool for modifying pages, storing the resulting state, and sharing it back through a link.

Word game

reWord

Sentence puzzle game built with Python, Google App Engine, and JavaScript.

Marketing product

Evangeler

Viral-marketing campaign product with Python, Backbone, Facebook integrations, and campaign mechanics.

Education game

Multiplication Master

Math practice game built with Python, Bootstrap, JavaScript, and Google App Engine.

Word games

Addicting Word Games

Word-game portal built with Python, Bootstrap, JavaScript, and Google App Engine.

Kids creative app

Joy Drop

Kids creative app built with JavaScript and Google App Engine.

Word game

Word Smashing

Word puzzle game across Python, Bootstrap, JavaScript, App Engine, and browser/mobile packaging.

Price comparison

Shopping Smart

New Zealand price-comparison product with ASP.NET, C#, MVC, ecommerce crawling, and marketplace-style data handling.

BSc Computer Science - University of Auckland

  • I completed Computer Science in two and a half years.
  • My record was A/A+ dominant across programming, algorithms, distributed systems, AI, graphics, operating systems, databases, and computational science.
  • I was a Teaching Assistant for Java, Python, and practical computing, and I genuinely enjoy teaching people how computers work.
  • First place in CS101 game programming; first place in CS320 fastest C quickselect; 6th in the 2012 New Zealand Programming Competition tertiary open.

Scholarships

  • Kiwiplan Scholarship for excellence in Computer Science and Software Engineering.
  • Top in the class Scholarship for COMPSCI 369 Computational Science.
  • J.C. Butcher Award in Theoretical Computer Science.

Certifications

  • Machine Learning, Deep Learning, Bayesian Learning and Model Deployment in Python.
  • Architecting on AWS.
  • Coursera Machine Learning.
  • Advanced Google Analytics.
Expand selected academic results

Academic highlights across programming, algorithms, distributed systems, AI, graphics, operating systems, and databases.

COMPSCI 101 Principles of ProgrammingA+ COMPSCI 105 Principles of Computer ScienceA+ COMPSCI 220 Algorithms and Data StructuresA+ COMPSCI 225 Discrete Structures in Mathematics and Computer ScienceA+ COMPSCI 230 Software ConstructionA+ COMPSCI 335 Distributed Objects, Services and ProgrammingA+ COMPSCI 369 Computational ScienceA+ PHYSICS 140 Digital FundamentalsA+ COMPSCI 215 Computer Systems 2A COMPSCI 320 Applied AlgorithmicsA COMPSCI 350 Mathematical Foundations of Computer ScienceA COMPSCI 367 Artificial IntelligenceA COMPSCI 373 Computer Graphics and Image ProcessingA COMPSCI 210 Computer Systems 1A- COMPSCI 280 Enterprise Software DevelopmentA- COMPSCI 340 Operating SystemsA- COMPSCI 351 Fundamentals of Database SystemsA- MATHS 108 General Mathematics 1A- MATHS 208 General Mathematics 2A- PHIL 105G Critical ThinkingA- STATS 101 Introduction to StatisticsB+ COMPSCI 314 Modern Data CommunicationsB EDUC 121G How People LearnB

Patent

Neural AI ensembles for image background removal.

Technical

  • PyTorch, Hugging Face, CUDA, Triton, ONNX
  • compression, distributed training, RL, FastAPI
  • Autoresearch, AWS/GCP, Docker, JAX, Terraform

Creative

  • Image/video generation and editing
  • VFX, motion, mocap, crowd, physics simulation tools
  • Games, 3D assets, graphics pipelines, procedural systems