Click on a project to read more!

Paddles Demo

Lua

Renoise API

A game for Renoise

Paddles

The Concept

Being a musician (drums, synths), one of my favorite programs is Renoise. After learning to program, I remembered that Renoise offers a Lua API to create extensions of the program! I decided to make a game within Renoise. Wanting something classic, I decided on Pong. The only other game made using the Renoise API is a port of Nibbles ("Nibbles" is a Snake clone included in FastTracker2).

Nibbles Gameplay

Renoise port of "Nibbles"

The Development

Learning Lua

I learned Lua using the official guide and documentation. I found my favorite feature of the language to be tables. Tables are so simple in contrast to C++'s structs, arrays, enums, vectors, hashmaps, etc. If you understand how they work under the hood, they can be very performant, too!

Engineering a Display

The Renoise API doesn't really offer a way to directly control pixels. Instead, it offers elements like sliders, knobs, text boxes, etc. However, I found that I could control a grid of tiny bitmaps (loaded from the filesystem) to create a "virtual screen"! Each frame, I had to manually update bitmaps, rather than blanking the whole screen and redrawing everything as graphics engines typically do, because Renoise isn't very fast at updating large amounts of bitmaps (it can update ~100 bitmaps at 25 frames/second).

Renoise Controls

Renoise ViewBuilder API controls & my bitmaps

The Launch

After some development, the game had a rainbow that trailed behind the ball, an AI opponent, a 2-player mode, sound effects, and more! Within 24 hours of its release, it was featured on the front page of the Renoise website's "Tools" section (where it still remains today), and even received recognition from Renoise's lead programmer and CEO; Eduard Müller (aka "taktik").

Early Paddles Demo

Early release of Paddles

The Update

Upgrading the Display

After the game's release, I discovered that Renoise provided another GUI element capable of creating a "virtual screen". Buttons can be scaled, and colored via a HEX color-code. This change offered a full 8-bit color display—capable of displaying 16,777,216 colors! To top it off, Renoise updates them 9x faster than bitmaps! I also implemented a double frame buffer system that only pushed updates to buttons that needed it. This allowed for more complex graphics, as I could now blank the screen each frame!

Button Demonstration

A button

Utilizing the New Display

I had been studying some water simulation algorithms for use in my 3D graphics projects, and decided to implement a water simulation in my Pong game for some nice visuals! The update was released and received more positive feedback from the Renoise community!

Paddles V3.0 Demo

Latest release of Paddles

Reflecting

If I were to continue working on this project, I would enhance the sound effects, and add a Breakout-style game mode. I would also like to write a 3D renderer for Renoise's Lua API someday, and create a small 3D game using it, though other projects are currently higher priorites.

Hopes

I sometimes worry that Renoise will cease development. Its keyboard–only tracker-style approach was common from the 1980's to the early 2000's, but most users today prefer a piano–roll interface. I hope that my Renoise tools will attract more like-minded users to the program, and keep its development alive for years to come.

Piano Roll vs. Tracker

The same MIDI sequence in a piano roll and a tracker

Curves Demo

Lua

Renoise API

Pixel-perfect
weighted
Bézier curvesn→∞

Curves

The Purpose

During development on Reform, the need arose for the calculation and rasterization of various curves. My first solution was to use logarithmic functions. They worked, but implementing discrete mathematical functions for different curve shapes wasn't a very robust solution. I ended up going down a rabbit hole of math and 2D rasterization that yielded the desired result. This project was the testing ground where I isolated this task.

The Development

Infinite-Degree Bézier Curves

After a lot of digging, I found Bézier Curves. Compared to Linear, Quadratic, and Cubic Bézier curves, I wanted my curves to support an infinite number of control points. I found this recursive definition of a Bézier curve. After implementing this—along with the associated functions for Binomial Coefficients and Bernstein Basis Polynomials—my curves supported an infinite number of control points! A very proud moment indeed!

Curves of Varying Degrees

Curves of varying degrees

Tension

I wanted each control point to have a variable "weight" (or "tension") to control its influence on the curve. I found an amazing resource called A Primer on Bézier Curves by Mike Kamermans (aka "Pomax"), which contains a chapter on adding this "weight"! After implementing this new math, I now had infinite-degree, weighted, Bézier curves!

Curve Tension

Adjusting weight

Rasterization

Bézier curves are rendered in a unique way. Rather than solving for X / Y coordinates per–pixel, you have to solve for t—which represents an interpolation between the curve's endpoints.

Showing t interpolation

t being interpolated

Naiveté

A naive approach is to just sample a high number of points, and fill the pixels where those points lie. The result of this approach is a curve that's too thick in high–tension segments, and breaking apart in low–tension segments. A better approach was clearly needed.

Curve sample sizes

Uneven t distribution

Pixel-Perfect

My solution was to sample a moderate amount of points, and connect them with line segments, which are easier to rasterize (which I later discovered is called "flattening" the curve). More robust solutions have been documented (such as in Chapter 4 of A Rasterizing Algorithm for Drawing Curves by Alois Zingl), but this solution would be sufficient, and quicker to implement. Wanting pixel–perfect, aliased lines, I found the Bresenham algorithm, which yielded a beautiful result!

Flattening the curve

Flattening a curve

Finished!

Being modular/loosely coupled, my finished code was easily implemented into Reform, with 2 curve types available now, and a custom curve editor to come in a future update. I will definitely be utilizing the knowledge from this project in other future projects as well! I can already imagine the possibilities of what can be created using these techniques!

Reform Curve Demo

Bézier curves implementation in Reform

Reform Artwork

Lua

Renoise API

A note transformation tool for Renoise

Reform

The Concept

After the initial release of Paddles, I got another idea for a Renoise tool; a tool that makes it quick and easy to powerfully fine–tune strums. The idea was inspired by FL Studio's "Strumizer" tool. It offers knobs and sliders to quickly and easily edit timings and velocities. You see the changes happen to the notes in realtime. Pressing the spacebar allows you to quickly hear the strum. This allows you to create and fine–tune strums very quickly.

FL Studio's Strumizer

FL Studio's Strumizer

The Problem

Being a tracker, Renoise represents musical notes as text on a spreadsheet. By default, each cell on the spreadsheet corresponds to a 16th note. To have notes trigger in–between this 16th–note grid, you must type a hexadecimal value 0x00–0x80 into the "delay" column next to the note. This results in an inefficient process for creating strums that are not in sync with the grid's time division, requiring the user to manually type individual hexadecimal values for each note over and over again.

Manual Strum Editing in Renoise

Strum editing in Renoise

My Solution

I decided to create a tool for Renoise that resembled FL Studio's Strumizer. Some priorites of the project included... a playful UI, intuitive controls, notes updating in realtime, easy preview playback, strumming across pattern boundaries, and more. This all added up to be quite a big project for a tracker–based environment.

WarioWare D.I.Y. Music Studio

WarioWare D.I.Y. inspired Reform's UI

The Development

It would take much too long to discuss every stage of Reform's development (be my guest to have a look at the source code though if you'd like), but I will cover some of the noteworthy stages of development. Just because something isn't discussed here doesn't mean it wasn't a significant stage of development, either.

Caching

To allow manipulated notes to overflow into other patterns, avoid colliding with other notes, and wrap from the end of the song to the beginning, I needed to know pattern lengths, song lengths, and more. When you're dealing with a large amount of notes spanning across multiple patterns, getting all of that data from Renoise's API becomes very slow. My solution was to cache the data Lua–side after retrieving it from Renoise. Each time I accessed data, I would first check to see if I had it cached, and if that cached data was valid. If the "valid" flag was false, I would get the data from the Renoise API. I would set the cached data's "valid" flag to false when changes were detected using the Observer–style notifiers that Renoise provides. This resulted in a dramatic increase in performance, while still ensuring that calculations are done with accurate data, that is only retrieved when needed.

Caching Diagram

Basic Caching Diagram

Curving

Through the development of my Curves tool, I studied Beziér curves, and implemented them in Reform to allow selections of notes to have their timings redistributed according to a curve. Note timings are treated as if they are distributed linearly upon selection, and can then be redistributed along a Quadratic Beziér curve, or an S-shaped Cubic curve. I could easily add more curves, and even allow users to create custom curves, but for the majority of use–cases, these two curve types are robust.

Reform's Curve Function Demo

Reform's Curve Function

The Result

This is my most organized, in–depth project to date, requiring a lot of learning, problem–solving, and optimizations to get the accurate, performant result that was achieved. I have some features I plan to add later on, such as undo/redo support, note–length preservation, quantize/humanize control, granular anchor placement, and a custom curve editor (based on my Curves editor). Despite this to–do list, I consider this project to be very full–featured and polished to a highly professional degree. I decided to provide the tool for free, out of love for Renoise and the many good times I've had with it.

Reform Demo

Demonstrating some of Reform's functions

SNEK Demo

C++

WinCon.h

FMOD

A game for the Windows Console

SNEK

Starting Out

This was my first substantial programming project. I chose C++ as my first programming language because I wanted a low–level foundation for my understanding of programming, and because I find working with computers at a low level to be fun and interesting (for these reasons, I've also dabbled in GBZ80 assembly, and the circuitry of GameBoy modding). I chose the Windows Console for my first project because it has a very simple API that would allow me to focus on learning fundamental programming concepts.

Rendering

The game was first rendered using cout statements, but this didn't result in a framerate–capable runtime (although it did have a cool film–like effect). So I learned and refactored my code to write directly to the console buffer using the WriteConsoleOutputCharacter() function, which resulted in a stable image capable of faster framerates. It also allowed me to easily add some color to the game, using the WriteConsoleOutputAttribute() function.

SNEK Old Version vs New Version

Rendering comparison: cout vs WriteConsoleOutput()

Sound

As the project matured, I wanted to add sound to the game, and decided to learn FMOD Studio and the FMOD Studio API. From this endeavor I learned how to read API documentation, which assured me that I would be able to learn and use any libraries I want for future projects. I did successfully add sound effects, as well as music that is synced to the player's movement (each frame that the snake moves corresponds to an 8th note in the music). The music also reacts when the player eats a fruit exactly on a downbeat, gets a new high score, and more.

Music from the game
(Song sections progress as
the player's score increases)

Reflecting

Now that I've matured as a developer, I can see there are areas in this project where code can be cleaned up, and more elegant solutions can be implemented for certain features. But, I've decided to leave it as–is for the time being, and learn new technologies with more potential for a wider impact. If I were to return to this project, my first priority would be to make it cross–platform with Windows, Linux, and Mac. Then I would add more gameplay features and visual effects.

HTML

CSS

JS

Three.js

This website!

aqu.surf

My Portfolio

For my portfolio, I wanted something fun and impressive, that shows my passion and curiosity for technology and design. I absolutely love 3D graphics, especially refraction effects, and especially water. I wrote the water simulation and flip cards in pure JavaScript. I'm using three.js to render the 3D scene after updating the water mesh's geometry each frame.

Adaptive Performance

To accomodate a wide range of devices, I'm constantly monitoring the frame-rate and lowering three.js's rendering resolution if the framerate dips below 60fps. This way high-end devices get a nice HD experience, and low-spec devices will still run smoothly after the renderer drops to a lower resolution.

Adaptive Display

I wanted a consistent density of indices for the water's geometry regardless of the aspect ratio of the current HTML document. To achieve this, I declare a constant that represents the desired amount of points. I then use the Quadratic formula to solve for the amount of points required in the X and Y axes that would result in a total area equal to the desired amount of points based on the current aspect ratio of the window. Because the water simulation is geometry-and-framerate-dependent, this process allows the water to move at a consistent speed regardless of the aspect ratio of the device or window being used to run it.

Art

I create all of my own sounds, 3D models, artwork, and code, for all of my projects. I like to have full creative control. When it comes to code I'm careful not to reinvent the wheel, but sometimes, to attain a deep understanding of a certain subject, I like to build things from scratch.

Results

I'm very happy with the way the website turned out! This was my first time writing HTML, CSS, and JavaScript, and I had a lot of fun with it and learned a lot about web development! I look forward to further developing my website! I think the website gives a pretty good representation of my interests and projects. If you have any questions or comments you can email me or message me on any of the platforms I have linked below! :)

About Me

Picture of me!

My name is James Graham! I love the beautiful experiences and versatile tools that can be created with technology!

Over the past 3+ years, I've found a huge passion for programming. I love solving complex problems, reading documentation, books, and dev-logs, and expanding my understanding! I love the amazing things that can be created when hardware, software, and creativity come together!

I enjoy lightweight programming with low-level languages like C and C++, and have even dabbled in Assembly (GBZ80). I also enjoy how fast and easy development can be when using high-level languages like Lua, Python, and even JavaScript. Despite C++ being my first language (and probably my favorite), most of my time programming has been spent in Lua, making tools for Renoise, a Digital Audio Workstation.

Some of my hobbies include: gaming (especially Nintendo games from Gamecube/DS and earlier), game dev, web dev, sound design & composition, drums, 3D art, pixel art, modding handheld consoles, inline-skating, bike-riding, skateboarding, swimming, and cooking.

If you want to talk, contact me on LinkedIn, AngelList, or by email! :)

Skills

C

C++

Lua

HTML5

CSS3

JavaScript

Three.js

FMOD

Unity

C#

.NET

Git

VS Code

Visual
Studio

Blender

Cinema 4D

Photoshop

Premiere

Aseprite

Renoise

FL Studio

LSDJ

GBZ80

Arduino

Learning

Godot

ZBrush

Neovim

Powershell

MSYS2

Links