synesthesia music software colors into music

NEW: Composition Software
Free & Open Source.
Coding MIDI Music for Raspberry Pi

CipherTune 暗号音楽

日本語Eブック / Japanese eBook
LiveCodeプログラミング 初心者開発入門

RGB MusicLabを作った人間の言葉に近いプログラミング

Sample RGB Musics
Composition FUKUSHIMA 2011 video

This project exhibited in FILE Media Art 2012, São Paulo.

Sample RGB Musics
One Year Web Project (01/01/11 to 12/31/11)
The latest browser has sounds problem.

This project exhibited in FILE RIO 2011, Rio de Janeiro and Web Biennial 2012 Istanbul Contemporary Art Museum.

RGB MusicLab Video Demo

RGB Music RENGA: musical improvisation

A series of photographs are converted RGB Musics.

New Yor City Subway

999 Views of Skyscrapers from Great Lawn

RGB Values into Music
converted to video work from RGB MusicLab file

String Trio Sphere Random Points
converted to video work from RGB MusicLab file

RGB Color Music Studies
The latest browser has sounds problem.

Subway Synesthesia / Kenji Kojima
Sound Installation of RGB Music
November 6 - December 13, 2008
AC [ Direct ]
547 W. 27th St, #519 New York, NY 10001

RGB Music 3D Space Player
It plays Internet files of RGB Music with 3D drawing of RGB points. Version 25 has this player.

Mona Lisa Scale Variations by Kenji Kojima
Studies of 7 varieties of scales. Chromatic, Melodic Minor, Harmonic Minor, Whole Tone, Gypsy, Blues and Okinawa.

Purple Green Variations by Kenji Kojima
Grand Piano, Top left: Thema Pattern and Colors, Top right: RGB invert, Bottom left: Green invert, Bottom right: Blue invert

RGB The Great Wave / 2:44 by Kenji Kojima
The Great Wave at the Kanagawa Oki by Hokusa

Earth, Time, Space / 6:32 by Kenji Kojima
The Earth from Apollo 17 by NASA

Antagata Dokosa / 1:35 by vo.Nagi & Kimux
It's Kimux's RGB Music "Itamemono" with vocal by Nagi. "Antagata Dokosa" is one of Japanese folk songs.

Composition of the Cut Off Seascape 4:04
by nagi

Four same Seacapes were jointed, then mosai cells were set 7, 7, 7, 8. After created RGB Music, the composer elaborated it in Apple GarageBand.

LEOPARD 3:43 by tio
Piano part was created by RGB MusicLab and mixed with others in Apple GarageBand.

RGB Experimental Music Laboratory / MacOSX and Windows Version
RGB実験音楽研究所 / MacOSX版 Windows版

This application was put together Kenji Kojima's RGB Music Project studies. RGB Music Project is a composite artwork of visuals, music and technologies that converts visual data into a music in attempts to discover relation between visuals and musics. The project core is the development of software RGB MusicLab.

RGB MusicLab converts RGB (Red, Green and Blue) value of an image and surface points (X, Y, Z) of 3D object to chromatic scale sounds. The program reads RGB value of pixels from the top left to the bottom right of an image. One pixel makes a harmony of three note of RGB value, and the length of note is determined by brightness of the pixel. RGB value 120 or 121 is the middle C, and RGB value 122 or 123 is added a half steps of the scale that is C#. Pure black that is R=0, G=0, B=0 is no sounds.

It is not an impression of a painting or a photograph of a musical variation. It is not an arbitrary process. It composes a score from an image directly. The technique of the music is called an algorithmic composition. Also this music and visual is a Multi-Media Art and sometimes refers to as Synethesia.

You can use a music which is created by RGB MusicLab freely. However it is under Creative Commons License.


RGBミュージックは、文化が培って来た感情を土台として作られる音楽ではないので、写真のイメージ通りの音は期待できません。RGB MusicLabを使って市販の音楽のようなものを作り出すと言うより、視覚も聴覚もコンピュータのデータでは同等に扱えるアートの可能性として、考えてもらうのが良いかもしれません。それは視覚と聴覚のような違う感覚を、コンピュータ上でミックスしたり分離したりもするアートに、発展できると言う意味です。

コンピュータの画像は沢山のピクセルからイメージを形作っていて、1つのピクセルは赤緑青(Red, Green, Blue)の光の3原色が数値で決められています。RGB MusicLab は、その数値を読み取って音階に変換します。RGBは 0 から255 までの数字が配布されていて、ほぼ中央の120をピアノの中央のド(ミドルC)として変換します。

アプリケーション「RGB MusicLab」の著作権は Kenji Kojima が所有します。RGB MusicLab で作った曲は、クリエイティブ・コモンズのライセンスにしたがってください。作った曲のコマーシャルでない配布、加工は自由です。細切れにして他の何かの素材として使ったり、ビデオやパフォーマンスのバックとしたり、自由に使用できます。右側に置いたサンプルが、音楽的に完成させるイメージを掴む、多少の参考になるかもしれません。あとは全てあなたの感覚しだいです。作成した曲のコマーシャルでの配布、使用は小島の許可が必要です。


Over $20USD donation for supporting Kenji Kojima's art works.
I will send a password of the desktop version "RGB MusicLab" to you in a few days.
The password is for a personal use only. You cannot give it to other people.
Please use the basic version before you donate. The basic has all functions,
but it can convert the first 10 colors of the image to a music.
Windows7 64bit machine cannot create a file.
Even if you cannot use the program, I will not refund your donation after I send a passowrd.
All rights of application "RGB MusicLab" are reserved by Kenji Kojima.
No password can play the first 10 colors of music.
A music created by "RGB Musiclab" is licensed under a
Creative Commons License.
MacOSX" "RGB MusicLab" V.41 Basic
Mac User: If you open this first time, "Control key + Click". Select the top menu "Open".
Small window asks "Are you sure open this Application?". Click "Open".

Windows: "RGB MusicLab" V.41 Basic
Windows can save a MIDI file only.

Quick Start:
Double click on the MonaLisa program icon. Program is opened. Drag-and-Drop your JPEG image onto the right image panel (1 on the top figure). Mosaic of the image is made on the left panel. Click on "Create RGB Music" (9 on the top image). Progress bar apears. Wait a short while. QuickTime controller apears. You can listen your music. If you like to save it to MIDI file, click on "Save Music" (10 on the top figure).

How to Create Your RGB Music

What's New in V. 41:
• No password can play the first 10 colors of music

What's New in V. 40:
• Fixed: In some cases "RGB_MusicLabData.txt" did not set the correct mosaic size.

What's New in V. 39:
• No password can play the first 10 colors of music

What's New in V. 38:
• Fixed RGB Music Player.

What's New in 37 & 37b (MacOSX Universal & Windows):
• First time opened V.37 it lost some new algorithmic data. V37b was fixed.
• Added Algorithm 2 (85keys) and Algorithm 3 (64keys). The original RGB Music alogorithm is 127 keys. Most of all people feel unpleasand sounds to very high tone notes and low tone notes. Algorithm 2 is that RGB value 3 is a half step note, and Algorithm 3 is that RGB value 4 is a half step note. All of Algorithms are the middle C is RGB value 120.

• Some adjustments for New Algorithmns.
• Added 4x4 mosaic (Windows)

What's New in 36 (Only MacOSX Intel):
• Can make 4x4 mosaic

What's New in 35:
• Fixed "Import RGB Music Data". V33, V34 did not show correct mosaic pattern.
• Fixed "Import mosaic to 3D (3D Palette)"
• Fixed Music Player (Only Windows version).

What's New in 35:
• Fixed: Importe RGB Music Data. V33, V34 did not show correct mosaic pattern.
• Fixed: Import mosaic to 3D
• Only Windows version: Fixed Music Player controller.

What's New in 34:
• Added separated RGB value of note length (not harmony length)

Left: Harmony note. Click on the top left icon. Right: separated RGB note.

• Added random selection of instruments.
• Fixed color range editor bug
• Fixed "Import RGB MusicLab Data..." bug

What's New in 33:
• Added text height tool.
• Improved Creating MIDI file can convert to Mosaic Image.
(Drag-and-Drop MIDI file which is made by RGB MusicLab onto Mosaic Image)
• Fixed Whole Tone Scale filter bug. (RGB Value 255 was converted to 0)

What's New in 32:
• Added paint and type tools.
• A Letter to Duchamp or How you can make a music letter.
• If RGB value 0 contains in pixels, MIDI file to mosaic image collapses.

What's New in 31:
• Bug fixed: "Reset Point > Start Point 120,120,120" of 3D pallet did not work.
• Bug fixed: "Import mosaic of Original Shape" of 3D pallet did not work

What's New in 30:
• Surface points of 3D object can be converted to RGB Music.
• Added 3D drawing tool pallet. Can import Waterfront obj file.
• Create RGB Music Sequence has 3D drawing animation.
• Added RGB Music RENGA in Theatre menu.
• Added new scale filters in Change Mosaic menu.

Click on 3D Drawing Icon. It opens a palette

3D drawing palette

What's New in 26, 26b & 26c:
• 26c Fixed: URL of Kenji Kojima's works in the Theatre.
• 26b Fixed: Double click on a list field in Theatre happened "Error 01" sometimes.
• Theatre menu > RGB Space Music... was Changed to "RGB Web Player...". It is a RGB Music player of your internet and local files. If you like to make "RGB Web Player" files, check mark on "Create Info Text" in "Create RGB Music Sequence (Theatre menu)". see fig yellow circle.

It makes "RGB Music.rev" and "RGB Music.txt" files when you export file at sametime. Put both files into your local folder or your Internet directory (like ). "RGB Music.txt" contains a title string and a mosaic data. When you open RGB Music list, type a Web address and hit return key (fig red square), or click on a folder icon (fig red circle) for local files. It makes RGB Music file list of the directory.

This handles two kind of servers.
Sample 1:

You can see the file list of index of rgbmusic/samples on a Web browser. You just put "RGB Music.rev" and "RGB Music.txt" files into the directory (folder).

Sample 2:
This server does not show a file list of index. see below.

If you server shows this kind of page, you have to make a file list of a html file. Please download a sample index.html file ( And put it into the directory. Index.html file makes a file list of "RGB Music.rev" and "RGB Music.txt". The page shows like below.

• Fixed: Change Mosaic > Open Color Sound Editor had a sound problem on V25.
• Fixed: Select Key and Custom Scale Windows did not open at the first time selection.

What's New in 25:
• Added: IntelMac and Windows can save AIFF file.
• Added: Moving Aim on Mosaic in Create RGB Music Sequence
• Added: RGB Space Music by Kenji Kojima in Theatre (RGB Music Theatre in V24 is No longer supported )

What's New in 24:
• Added: Select Key. There are two "Select Key" 1) Change Mosaic > Scale: Adjust Half Step > Select Key and Custom Scale... 2) Change Mosaic > Scale: Replace to No Sounds > Select Key and Custom Scale... "Select Key" is not a change of key in musical terms. When you make a scale of image, you can select a start key of a scale. Default scale keys before version 23 were C. Now you can set any keys and can make any custom scales.

What's New in 23:
• Fixed: When you selected "Paste Image into Mosaic" from "Change Mosaic" menu on version 22, the program froze.
• Changed: The largest image size of "Paste Image into Mosaic" is changed to 400x400.
• Added: "Default Setting" in File menu.

What's New in 22:
• Added: Double Harmonic Major Scale and Double Harmonic Minor Scale in the Change Mosaic menu.
• Added: Show Title in Create RGB Music Sequence settings
Fixed: Second instruments appeared on the Theatre.
• Fixed: Second Instruments change keys affected no sounds note (RGB value 0).
• Fixed: Some of settings were not saved in RGB MusicLab data.

What's New in 21:
Added "Second Instruments". You can set up to 6 instruments. Second Instruments have changing keys. Plus 1 is a half step up, and minus 1 is a half step down. The setting is from -60 to +60. An octave up is +12.

Color Range Editor has unlimited "Back to Preview".
• Fixed Pause and Play button icon in RGB Music Theatre.
• Fixed Paste into Mosaic.

What's New in 20:
Added "Theatre". You can think it is a kind of animation player for RGB Music Color Sequence. It plays a synchronized music and colors. See figs. The Theatre can be opened from menu “Theatre” of menu item “RGB Music Theatre”, or “Theatre Icon” which is under the right image panel. RGB Music Color Sequences play on the Theatre through Internet. If you like you can register your site to the Theatre programs. There are Kenji Kojima's two sites in the program now. Click here How to make the site.
• Added menu item “Create RGB Music Sequence”. It creates a RGB Music Color Sequence document. Created documents must have extension ".rev".
• Added “Open URL...”. It opens a RGB Music Color Sequence document. which is on the Internet.
• Added “Open Local File...”. It opens a RGB Music Color Sequence document from your local disk.
• Improved paste into mosaic.
• Fixed some bugs.

RGB Music Theatre: Opening (left), Playing (right)
If you like to show a menu bar or a task bar double click on the background color.

What's New in 19:
A cursor changes to a drag-and-drop icon when you can drop an image.
• Mosaic can be copied and pasted through clipboard directly. Paste size must be under 40 x 40 pixels.
If you copy an image in PhotoShop, the image has to be 100%. Otherwise you cannot set a correct pixel size, when you paste into a Mosaic. (Change Mosaic menu, Copy Mosaic, Paste Image into Mosaic)
• PING snapshot can change size. Under 800 x 600 pixels. (Camera icon menu, Snap Mosaic Image > PING Image Size... )

• Mosaic pattern can be color sequence files. (Camera icon menu, Export PING Color Sequence...)
• Improved Rotate, Flip Mosaic and some bugs fixed.

What's New in 18:
Added: Okinawa scale (C, E, F, G, B) in "Change Mosaic" menu. Okinawa is southern islands in Japan. They have their own musical scale. It is close to Javanese scale. Listen the sample in "Mona Lisa Scale Variations".
Added: 5 Exchange RGB colors in "Change Mosaic" menu. Functionally it is same as exchanging instruments of RGB color, but it is modified from a color score.
Added: 7 Invert RGB colors in "Change Mosaic" menu.

What's New in 17:
Added: PMD (Play Midi Data) script editor. Create RGB music first, then open the editor. Click on the "C" on document icon at the right side of the window. (fig. right: small red circle) There is a PMD grammer and samples palette in the editor. You can modify your RGB music by the script like "C4 D E F G A B C5" and numbers. The basic grammer of the script is HyperTalk play command created by UDI.
• You can change the last note length of your RGB Music. (fig. right: large red circle)
• Now you can save a whole data of the RGB MusicLab window into one document. Save changed mosaic data was separated from a RGB Music data. Click on the Mona Lisa on document icon.
• Improved "Donator code". You don't need to type a donator code at the every update. It's in the preferences folder.

What's New in 16:
Change Mosaic menu has one time "Undo".
• "Snap Mosaic Image" has PING format. JPEG makes uneven colors on a mosaic cell.

What's New in 15:
Scales are added: Pentatonic Major, Pentatonic Minor, Pentatonic Blues, Whole Tone are in "Adjusted Half Step and Replace to No Sounds". Custom Scale setting is only in "Replace to No Sounds". And Grayscale is also added. Please see "Change Mosaic" menu(fig. right).
Added "Out of the range colors are No Sounds" in Color Range Editor. Too low and too high sounds are cut by this setting. Image pixels of no sounds will be closer to a primary color or black.
• There are two Panpot selections (fig. Left). Left radio button keeps same panpot values of each RGB score from the beginning to the end. It is set by a sliding bar. Value 0 is the most left, 10 is the most right, 5 is the center. Right radio button changes Panpot setting by a pitch tone. A lower note panpot is left and higher is right. Probably this setting is good for three same instruments like a piano solo. When the right radio button is selected, a sliding bar does not work, and you can not set "Create GarageBand 4 MIDI"..

About Scales
Major Scale: C D E F G A B, Natural Minor Scale: C D D# F G G# A#, Melodic Minor Scale: C D D# F A B, Harmonic Minor Scale: C D D# F G G# B, Pentatonic Major: C D F G A C, Pentatonic Minor: C D# F G A# C, Pentatonic Blues: C D# F F# G A#, Whole Tone: C D E F# G# A#, Gypy Scale: C D D# F# G G# B. You can set other musical scales using "Custom Scale Setting"(fig. above).
Grayscale is a visual art term. It is a range of gray shades from white to black. In the RGB MusicLab a pixel has same three RGB values that color is gray. RGB value "120,120,120" is a gray pixel and center C of the musical note. It is three notes but it is mono-tone, if you set three same instruments.

What's New in 14:
Bug fixed: Revert data was saved in some conditions instead of scale data.

What's New in 13:
Mosaic color can be changed by a selected scale. Scales are Major Scale, Natural Minor Scale, Melodic Minor Scale, Harmonic Minor Scale and Gypsy Scale. You can select from "Change Mosaic" menu them. There are two kinds of selections. "Adjust Half Step" is not applicable RGB values are adjusted a half step up or down, it depends on the note. "Replace to No Sounds" is not applicable RGB values are replaced to no sounds that is RGB value 0. They will be same as rests in the music. Mosaic color will be closer to a primary color or black. Please listen the sample music "Gypsy Smile".

What's New in 12:
Velocity setting has more variations: Select "Range" or "Meter" first. You can set any number of 10 to 0 velocity. When you set "Range", RGB MusciLab assigns the first value to next value on each pixel automatically by its brightness. When you set "Meter", Intervals of Notes enable. If you set same notes length and Intervals "3, 3, 3, 3", it takes regular triple time. And you set Intervals "3, 3, 3, 5", it takes irregular rhythm.

The Smile / 0:57 by Kenji Kojima was "3, 5, 3, 3 (Red)", "3, 3, 5, 3 (Green)", "3, 3, 3, 5 (Blue)". Each color music was taken separately, and combined.
• You can set no line breaks.
• Bug fixed: Rotate 90CW and 90CCW did not have correct line breaks.

What's New in 11:
Added Modulation. It is a vibration effect from 0 to 10. You can find it the right of Acoustic setting.
• Added Meter that is a rucurring pattern of velocity setting. You can select "Range" or "Meter" of velocity. If you select "Velocity: 10-6", "Intervals of Notes: 5" and "Meter". The first note is velocity 10, and other 4 notes are velocity 6. Probably you can recognize the differences clearly when you select "Every Notes is same length". Please listen a sample which I made. The tile is X_Plants / 2:34.

What's New in 10:
RGB Diagram was added in Color Range Editor. It shows mapping and quantities of RGB values in an image.
Bug fixed: Right after drag-and-drop a new image, color range editor did not work correctly.

What's New in 09:
Color Range Editor can adjust too low or too high sounds in the range setting.
Select "Open Color Range Editor" from "Change Mosaic" menu (fig. left). Color Range Editor opens (fig. right). There are R, G, and B color range controllers. The lowest RGB value is 0, and the highest value is 255. You can adjust RGB color ranges between them. You can recognize two values, one is number of RGB, another is keynotes like C0, C2, C4. Center C is RGB value 120 or 121, and keynote C4.

Selection "Push into the Range" is, if a color value is out of the range, it's changed the lowest value or the highest value. But colors which are inside of the range are not changed. "Proportional Arrange" is whole color values are shrunk in the range proportionally.

Changed color range (fig. left). Too low and too high sounds are adjusted.

• Added: "Create for GarageBand 4 MIDI". Apple GarageBand could not import a MIDI file of version 8. GarageBand 2, 3 can import version 08 files. If you like to create a MIDI file for GarageBand 4, check mark it. But the MIDI file cannot be converted to the original mosaic image.

What's New in 08:
• MIDI file which was created by RGB MusicLab can be converted to the original mosaic image. It demonstrates a RGB Music is converted true RGB color data.

Drag & Drop a RGB Music MIDI file onto the mosaic image. If you like to stop, type "Command key + Period" on MacOS, or "Control key + Period" on Windows.
MIDI file must be made by Separated RGB Parts, otherwise it does not show correct colors.

What's New in 07:
• Color/Sound Editor can change a color of a mosaic cell. Select "Open Color Sound Editor" from "Change Mosaic" button menu.

Listen Sound: Double click a color square. Ctrl key + Click on a color square -> Open color palette. and change clicked color.

• Mosaic Image shows RGB value by mouse moves.
• Click on a mosaic: Listen sounds of the color.
• Fixed some bugs.

Version 06b (Only MacOS):
• Fixed: "Save Music" did not work on Intel Mac. Intel Mac can save MIDI file, Power PC can save MIDI and AIFF file.

Version 06:
• Fixed: Intel Mac version. it could not created and save RGB Music.
• Added Change Mosaic: Flip Mosaic Horizontal, Flip Mosaic Vertical, Rotate 90 CW, Rotate 90 CCW, Ascending by Red value, Ascending by Green value. Ascending by Blue value, Descending by Red value, Descending by Green value, Descending by Blue value.
• Added acoustic effect of instruments.
• Changed tempo from 20 to 800
• Tempo can be changed by typing number.

Version 05:
• You can set a range of velocity
• Random cell mosaic
• You can drag and drop an image from Safari(MacOS) or IE(Windows) onto the right panel directly. But the image has a link info, RGB MusicLab rejects it.

Version 03:
• Progress bar shows.
• Export and import a data of RGB MusicLab by text file.

Version 02:
• Three menu items added in Snapshot.
• Image file name location is changed to under the original image.
• Showed disable note length icon.

How to Create Your RGB Music

There are additional information. Please read "What's New" too.

1. Drag-and-Drop your image file onto the right panel, or open an image from the file menu. You can use JPEG, PING and GIFF images. They are not only photographs, but drawings of these file formats. Also You can drag and drop an image from Safari(MacOS) or IE(Windows) onto the right panel directly. But if the image has a link info, RGB MusicLab rejects it. And you can drop a text data of RGB MusicLab.

2. A mosaic image of the original you drop is made to the left panel. The number of cells of the mosaic is shown at the bottom of the mosaic. RGB MusicLab converts a music from this mosaic image. You can change a large size cell or fine size cell of the mosaic. If you set a too fine size cell, you have to spend a long time for the conversion. Try large size first.

3. Select "RGB harmony" or " Separated RGB Parts" and a musical instrument. "Use Panpot" is the sound is left or right. And instruments can be set an acoustic effect. If you plan that you will put a MIDI file into another program later such as Apple GarageBand, you may not need the Panpot setting. The right of Panpot are, Acoustic and Modulation(vibration effect).

4. Set the tempo of the music and the note length. RGB MusicLab prepares three kind of note length. The first one is every note is same length. If you select this note icon appears and select the note. The second is dark pixel is shorter length, and the third is light pixel is shorter length.

5. RGB MusicLab reads pixels from the left to the right. You can put a rest in each line breaks. Select a rest symbol.

6. Select note length.

7. You can set a range of velocity. Velocity 10 is the strongest sound that is the darkest pixel. If you select "Velocity: 10 - 5", the lightest pixel is Velocity 5. When you select "Meter", Intervals of notes enables. 5 Intervals are the first note velocity is 10, and other note velocities are 5.

8. After you made a mosaic by a slider, you can change the mosaic to Flip Mosaic Horizontal, Flip Mosaic Vertical, Rotate 90 CW, Rotate 90 CCW, Ascending by Red value, Ascending by Green value. Ascending by Blue value, Descending by Red value, Descending by Green value, Descending by Blue value.

9. Click on "Create RGB Music". The cursor changes a busy icon during the conversion. If you like to stop it, type "Command key + Period" on MacOS, or "Control key + Period" on Windows.

10. Click on "Save Music". You can save the file to MIDI (MacOS and Windows), or AIFF (MacOS Power PC only). If you are IntelMac or Windows user and want to create AIFF files, put a MIDI file into Apple iTunes, then you can convert it to an AIFF file.

if you use MacOSX 10.5 Leopard, Save dialog layout may be broken. Click a triangle of "Save As". You can get a regular save dialog. See below. And select from "Export:" menu, "Music to Standard MIDI" or "Music to AIFF".

11. If you like to save the mosaic image and original image, click on "Camera" icon. And select a menu. A JPEG snapshot is made.

12. You can export a RGB MusicLab data by a text file. Click on the Mona Lisa document icon. You can import or drag and drop the data file.

13. Drag & Drop RGB MusicLab MIDI file onto a mosaic image, it is converted to the original mosaic image.

Email to Kenji Kojima