Drag a sticker or use the arrow keys to turn it. Home restores the front view.
Speech, keywords and accents
Experiments2020-2021
Keyword spotting: an example from my PyTorch implementation.QuartzNet architecture. Kriman et al., 2019, Fig. 1.
I implemented QuartzNet speech recognition from scratch in PyTorch, alongside attention-based keyword spotting and an accent-classification team project.
I tested whether expression trees helped Transformers integrate expressions and solve differential equations. The idea did not beat the baseline; the evaluation anomalies were the interesting result.
Giving the model explicit expression-tree structure did not improve the baseline overall. In a small set of examples with long numbers, relative methods handled the digits better: the baseline could mix their order, while structure-aware attention often found the correct answer among its first ten hypotheses.
I fine-tuned ruDialoGPT3 on Telegram exports to make a bot that talked like me. A personal funny thing about the result: it was depressed! As I was back then!
Received dozens of GitHub stars. Open source is the way!
Project2021-2025300k+ Hugging Face model downloads
Teaching attention, before implementing it.
The actual input
Timur:@summarization_bot
Timur: Does it work?)
Timur: Yes)
Timur: Pretty funny
Timur: I don't have any text handy
Timur: Too many requests!
Timur: I didn't understand how it works
Timur: It needs to be added to the chat
Timur: Or forward it messages from the chat
Here's what came out:
Timur is surprised that there is a so-called "summarization_bot". It needs to be added to the chat or sent messages from the chat.
An actual chat and the bot's summary.
300k+Hugging Face model downloads1,000+daily requests, historically$100+/monthhistorical revenue$6,000investment won in an HSE competition
I taught three school pupils self-attention, then we implemented Transformers in PyTorch and built a Russian chat-summary bot. The project ran from 2021 to 2025 and won $6,000 in investment through an HSE competition. Keeping it going became my first experience managing a team, subscriptions and server bills.
From the first lesson to closure / model, sources & note
We started before ChatGPT. I kept the project going after the original intensive program, designing features around user requests. Our teaching implementation was pure PyTorch; the deployed summarization model was a fine-tuned mBART, which remains available.
The example shows a chat about using the bot: adding it to a group or forwarding messages. The project became financially self-sustaining, but that didn't last. When it closed, subscriber support covered less than half of what we needed to break even.
I built LLM memory and retrieval systems for EVA AI. Integrating conversation summaries gave the chatbot 3x longer memory and increased ARPU by 9%.
~500kMAU / ML team of <5$25k/daytotal app revenue at the time3xlonger memory+9%ARPU from summarization
EVA AI today: official App Store image, October 2026.
Dialogue summarization
I integrated summaries to retain more conversation: 3x longer memory and a 9% increase in ARPU.
Questions about yesterday
Generating questions about the previous day's dialogue increased ARPU by 8%.
Fact extraction and retrieval
Built from scratch RAG-based (Retrieval-augmented generation) pipeline for extracting and using facts by a custom LLM. Used vector DB and scaled it for half a million monthly active users.
Automated error classification
I automated collection, labeling, training and deployment. Unethical responses fell by 35%, as measured by OpenAI's safety classifier.
A smaller classifier
A distilled, quantized BERT-like classifier replaced rules and improved quality at roughly the same throughput.
The engineering behind the numbers / sources
Example of a task: Llama 2 could reply in other languages, but our English-only classifier missed messages that should have been billed. A multilingual encoder and a head trained on existing English labels fixed it in a day.
For a Mercedes-Benz and Salesforce challenge, I built an EV sales assistant with Denis and Anastasiia using specialist agents, retrieval and a supervisor. Second among more than 80 teams.
A review connecting HiPPO, linear recurrent models, and attention/state-space duality.
3D reconstruction from point clouds
Poster2025 / TUM ML43D
With our Convolutional Occupancy Networks poster.
Noisy point-cloud reconstruction with Anastasiia Chaikova, Omar Al-Dabooni and Yi Wang: sinusoidal activations, positional encodings and semantic conditioning.
My TUM master's thesis with JetBrains asked whether a coding agent could carry less context between steps. I used an encoder to compress tool output into fixed-size memory embeddings for the language model. The compressed memory retained answers on single-shot tasks, but agents solved fewer software-engineering issues.
The follow-up paper investigates two possible reasons for the multi-step performance drop: errors accumulating over time and important details being lost during compression.
This poster records partial research on choosing a model for each step of a software-engineering agent. Early prefixes provided weak routing signals; training raised problems with exploration and the cost-quality tradeoff.
Qwen3.5-4B / 4k training cap / AIME 2024/25 avg@3 / against the base model at each test budget.
The initial idea was mine: what does a fixed training budget teach a reasoning model? I noticed that giving trained models more tokens could put them behind the baseline. We investigated that reversal.
Full paper / coming soon
Setup, limitations, poster & sources
Training improved answers near the training cap, but could weaken the ability to benefit from more test-time tokens. Evaluating only at the cap would have hidden the reversal shown above.
The study covers three small model families, with one training seed per condition. Shorter responses remained useful on some transfer tasks. Keeping the gains without losing high-budget performance remains an open problem.
