Introduction: In search of a semantic timeline
For every productivity freak, there’s a universal problem: the gap between a tidy calendar and the chaotic reality of a workday.
I would know - traditional time-tracking tools give you data - “3 hours in Chrome” - but what am I supposed to do with that? On the other hand, manually jotting down what you did in specific time slots is no less problematic. This fundamental gap in understanding our own productivity, for those who feel an inner need to close it, creates an opportunity for Dayflow - a project that aims to redefine how we perceive and analyze our screen time - for the good or the bad.
Dayflow is not yet another time tracker - it’s an ambitious attempt to build a “semantic timeline,” or, to use the project’s own metaphor, “a git log for your day” (you can tell it was built by nerds like us). The app automatically documents what the user is actually working on by analyzing and understanding the content displayed on the screen. Instead of raw metrics, it provides a narrative of the workday, transforming the stream of activity into a series of meaningful “commits.”
The project’s creator is JerryZLiu, a software engineer with experience at companies like Flexport and Google. Dayflow is built on core principles that shape its entire architecture: privacy as a priority, full user control over data, and transparency afforded by open-source code under the MIT license. It’s a deliberate, direct response to the inherent privacy concerns raised by an app that continuously monitors your screen.
In this piece, we’ll deconstruct Dayflow’s architecture, dig into its most fascinating engineering patterns, and take a look at how it handles security and privacy. We’ll examine how a clever concept was translated into a working, efficient, and—crucially—trustworthy product.
Dayflow’s Philosophy
The core problem Dayflow tries to solve is the inadequacy of traditional productivity-tracking methods. App-based metrics are inherently superficial. As the creator notes, the goal is to move from a useless statement like “Chrome: 3 hours” to specific, comprehensible actions: “Reviewing PR comments: 45 min,” “Reading a Rust thread on HN: 20 min,” “Debugging auth flow: 1.5 hrs.” It’s a fundamental shift in perspective - from where time was spent to what was accomplished.
It’s worth noting that the idea of continuously recording life, known as “lifelogging,” isn’t new. Past attempts relied primarily on dedicated hardware such as wearable cameras (e.g., Narrative Clip) or smart glasses. These devices aimed to passively document the user’s entire life, often 24/7, capturing both professional and private interactions. Dayflow consciously departs from this model. Instead of relying on additional physical devices, it takes a purely software approach focused exclusively on the digital sphere of the work day. It doesn’t record conference-room meetings or coffee breaks; it precisely analyzes what happens on the computer screen, offering a more targeted and less invasive tool for reflecting on professional productivity.
The project draws inspiration from tools like Rewind, but its goal is fundamentally different. While Rewind focuses on information retrieval (“search your past”), Dayflow centers on reflection and course-correction in real time (“see what you actually spent time on”). This subtle yet crucial difference in product philosophy shifts the emphasis from a passive archive to an active tool for self-awareness and improving effectiveness.
A central pillar of Dayflow’s philosophy is an uncompromising approach to privacy. The creator builds the project on the conviction that “anything that watches your screen all day should be completely transparent about what it does with that information.” This principle isn’t an add-on — it’s the primary design constraint that dictates the entire architecture. It leads directly to the key decision to prioritize on-device processing. Dayflow thus becomes a manifesto against the trend of centralized, cloud-based AI tools, promoting data sovereignty and on-device intelligence as the default, not an option. By foregoing a built-in LLM in favor of interfacing with user-run tools like Ollama or LM Studio, the app hands even more control back to the user, reinforcing the fundamental principle of data ownership.
Architecture: A Five-Stage Pipeline for Daily Reflection
At first glance, Dayflow’s architectural elegance doesn’t stem from a complex class hierarchy or intricate modules, but from a rigorously defined, cyclical data-processing pipeline. The system relies on a simple yet powerful stack: a native SwiftUI app, a local SQLite database managed via the GRDB library, and macOS native APIs for screen capture. The entire process can be divided into five logical, sequential stages that transform raw pixels into an orderly, semantic timeline.
Stage 1: Capture
The process starts with continuous screen recording using the modern ScreenCaptureKit framework. A key architectural decision is the deliberate limitation to just 1 frame per second (1 FPS). This video stream is then split into small, 15-second chunks that are temporarily saved to disk. This stage underpins the app’s overall efficiency, drastically reducing resource usage.
Stage 2: Analyze
Every 15 minutes, the system collects the accumulated 15-second chunks (forming a batch of 60 frames) and prepares them for analysis by the AI model. This batching strategy is a smart compromise between the need for timely feedback and the need to avoid the constant heavy load of nonstop model invocations.
Stage 3: Generate
The prepared batch of images is sent to a vision-language model (VLM). This can be a remote API like Gemini or a local model run by the user via Ollama. The model’s task is to visually analyze the frames and return a structured JSON response with a concise summary of the user’s activity. This is the heart of the app — the moment when raw visual data acquires semantic meaning.
Stage 4: Display
The AI-generated activity data is stored in the local GRDB database. The user interface, built with SwiftUI, is inherently state-driven. It observes changes in the database and, automatically, without imperative refreshes, renders the timeline and shows new activity cards. This ensures a smooth and responsive user experience.
Stage 5: Cleanup
A background process automatically removes raw recording files older than a defined period (3 days by default). This mechanism is crucial for disk-space management and ensures the app stays lightweight, occupying a predictable, limited amount of space (about 5 GB for 72 hours of recordings).
Patterns and Techniques
Behind Dayflow’s simplicity lies a set of well-considered engineering decisions and elegant patterns. Analyzing these techniques reveals a mature approach to software development that balances performance, flexibility, and privacy.
Pattern 1: Hyper-Efficient Capture with ScreenCaptureKit
The most important architectural decision enabling the product’s philosophy is the 1 FPS screen-capture strategy. This choice makes continuous, local 24/7 recording possible without crippling the system. Compared to standard 30 FPS recording, it reduces data volume by 99.7% - an astronomical figure.
The implementation relies on Apple’s modern ScreenCaptureKit framework, which offers far greater performance and flexibility than older APIs like AVFoundation. Although the source code isn’t directly available for analysis, we can reasonably infer how it’s done. The key is configuring the
SCStream object by setting its minimumFrameInterval property to one second. This instructs the system to deliver a new frame no more than once per second - far more efficient than capturing at a high rate and throwing away excess frames.
The simplified code snippet below illustrates how Dayflow configures the stream:
Pattern 2: A Two-Brain AI Engine - Cloud vs. Local
Dayflow gives users a choice: higher-quality analysis via the Gemini cloud model or maximum privacy via a local VLM run through Ollama. This flexibility, however, poses a serious engineering challenge. The creator openly admits how hard it was to adapt the app’s logic from powerful models like Gemini to much smaller local models like Qwen2.5VL-3B.
The key lies in advanced prompt engineering and data chunking. As disclosed, processing a 15-minute segment (60 frames) requires about 32 separate calls to the local model, compared to just 2 calls to Gemini. This implies a sophisticated orchestration layer that:
- Fetches the batch of 60 frames.
- Splits them into small groups (e.g., 2 frames each).
- Sends each small group sequentially to the local Ollama server on the
/api/generateendpoint. - Maintains context across those 32 calls, likely by passing a summary from the previous response as part of the next prompt.
- Finally synthesizes the 32 partial responses into a single coherent activity card.
Integration with Ollama is done via a simple REST call. Images must be Base64-encoded and attached to the JSON payload of a POST request.
Pattern 3: High-Performance Persistence with GRDB
The database choice is another example of a mature engineering decision. The creator deliberately rejected Apple’s default Core Data in favor of GRDB. Why? They needed complex time-range queries using window functions. Core Data, being an ORM, heavily abstracts SQL, making such operations extremely difficult or even impossible to implement efficiently. GRDB, while providing a safe, “swifty” interface, fully exposes the power of raw SQL.
As you can see, SQL window functions are a powerful tool, ideal for analyzing time series such as user activities. They allow computations over a subset of rows related to the current row. For example, to implement the “activity grouping” logic mentioned by the creator, you can use the LAG() function to fetch the end time of the previous activity without a complex SELF JOIN. This enables efficient merging of consecutive, similar actions into one longer card or detecting breaks in work.
Pattern 4: A Modern, Declarative UI with SwiftUI
Choosing SwiftUI as the UI framework is a natural consequence of a data-driven architecture. In SwiftUI, the UI is a direct function of application state. In Dayflow’s case, that state is the list of activities stored in the database. When the AIManager processes a new batch of data and adds a record to the database, the ViewModel observing those changes automatically notifies the view, which then re-renders to display the updated timeline.
The likely data flow uses modern SwiftUI patterns such as @Observable objects. A central TimelineViewModel would be responsible for fetching Activity records from a DatabaseManager. The main view, TimelineView, would simply observe this ViewModel and render a list of ActivityCardView components based on the provided data. This declarative approach eliminates manual UI manipulation, leading to more predictable, less error-prone, and easier-to-maintain code.
A Critical Look at Security and Privacy
For an app that, by definition, has access to everything a user does on their computer, security isn’t just another feature - it’s the foundation of its existence. Every architectural decision must be evaluated through the lens of potential threats. The analysis below critically assesses Dayflow’s security posture from multiple angles.
The first and fundamental security feature of Dayflow is its open-source model. Publishing the full source under the MIT license is a deliberate act of building trust, allowing the community to audit the code and verify that the app does exactly what it claims, with no hidden telemetry, unauthorized data exfiltration, or other malicious behavior. In a world where user data is a commodity, transparency is the strongest currency.
The most important privacy-preserving capability is the option to run the entire analytics pipeline 100% locally. By allowing users to connect to their own instance of Ollama or LM Studio, Dayflow guarantees that sensitive on-screen data — images, text, interactions - never leave the user’s machine.1 This eliminates an entire class of cloud-processing risks: provider server leaks, unauthorized employee access, or using data to train global AI models.
For users who opt for the higher analysis quality offered by the Gemini cloud API, secure storage of the API key is crucial. The creator states that the key is stored in the system Keychain. This is best practice in the Apple ecosystem. Keychain is a system-level encrypted database for storing secrets. Using the Security framework lets the app save, query, and read data such as API keys without storing them in plaintext on disk (e.g., in UserDefaults or config files).
The 15-second video fragments are temporarily stored on disk before processing and then deleted. This introduces potential risk. An attacker with filesystem access could capture these raw video files in the brief window between write and deletion. The risk is partially mitigated by the short retention period. However, the files’ location and permissions are critical. The app should write this data inside its sandbox, in an app-specific directory with restrictive permissions to block access by other processes.
Conclusions: The Future Is Semantically Understandable
Analyzing Dayflow offers invaluable lessons about the future of software creation in the age of AI.
Dayflow exemplifies a new paradigm in building software. The engineer’s role evolves from writing implementation code to orchestrating intelligent agents, defining clear data pipelines, and - most importantly - building robust systems that respect privacy and user control in a world increasingly hungry for data.
The creator’s future plans, such as “distillation” (using high-quality Gemini outputs to train and improve the local model) and fully configurable dashboards, point to a clear development path. These steps have the potential to narrow the quality gap between local and cloud processing while increasing the usefulness of the collected data.
Deserved star from me - and I think I’ll risk installing it after this analysis 😁
