GUI draw-ups on Google Docs

GUI draw-ups on Google Docs

Some ideas added to my Google Docs folder where they are available for viewing. Updates include the ‘main’ tempo control screen, filter screen and button interactions

First video of progress and Github repo created!

Update 1 video:

Github repo: https://github.com/freshfunkee/KinectMusicController

Update

Separated OpenCV functions and operations into it’s own class, “CVImage”. KinectController class now only operates on the NuiApi, CVImage handles all the OpenCV work such as building a frame matrix and Gui retrives the OpenCV frame and displays it on an SDL surface.

I also edited the project structure and I started using version control with git.

Progress?

In short, yes. Tracking of left hand is now implemented. Found the source of a memory leak and realised explicit destructor calls are not required.

I’m feeling very lazy today, so that’ll do. I’ll do some more tomorrow…. or the next day

v0.1 Completed

It took longer than I thought, but I have the basic elements of the project functioning. I’ll be going back to planning in the mean time to plan the next features to incorporate.

I might try to get a screen capture of the application for reference instead of posting code, since the line count and class files are growing.

MusicController_v0_1

Today, I am going to start my first attempt at the Kinect Music Controller. I intend to incorporate tempo control using the Kinect Sensor, whilst trying out my Song class by parsing an xml file for parameters. The idea is to see how all the different aspects of the project will work together. So far I have been developing each feature separately, so this should give me some ground to stand on and allow me to incorporate further features.

At the most basic level, the application will allow the user to select a song from the command line, use this to create a song class, play the song and allow the user to control the tempo of playback.

Tempo control of wav files with pitch correction in FMOD and C++

There aren’t a lot of references for tempo control with pitch correction online for FMOD although it does have the ability to do so. This process is based on 2 functions in FMOD. setFrequency(), which is the tempo control and DSP_PITCHSHIFT, which by varying the pitch parameter, can correct the unintentional pitch change when changing the frequency.

The code gets the deafult frequency for the wav stream, (in most cases 44100Hz), takes a user input for a new tempo, and sets the pitch parameter as a fraction of the default frequency over the user input.

#include "stdafx.h"

void errchk(FMOD_RESULT r)
{
	if(r != FMOD_OK)
	{
		printf("FMOD_ERROR! (%d) %s\n", r, FMOD_ErrorString(r));
	}
}

int _tmain(int argc, _TCHAR* argv[])
{
	FMOD::System *system;
	FMOD::Channel *channel=0;
	FMOD::Sound *stream1;
	FMOD::DSP *pitch;

	unsigned int version;
	float defreq, spfreq, pit;
	bool done = false, active = true;

	FMOD_RESULT result;

	result = FMOD::System_Create(&system);
	errchk(result);

	result = system->getVersion(&version);
	errchk(result);

	if(version < FMOD_VERSION)
	{
		printf("Error! You are using an old version of FMOD %08x. Required: %08x.", version, FMOD_VERSION);
		return 0;
	}

	result = system->init(1, FMOD_INIT_NORMAL, 0);
	errchk(result);

	result = system->createDSPByType(FMOD_DSP_TYPE_PITCHSHIFT, &pitch);
	errchk(result);

	result = system->createStream("C:/trackStreams/Balcony.wav", FMOD_HARDWARE | FMOD_LOOP_NORMAL| FMOD_2D, 0, &stream1);
	errchk(result);

	result = system->playSound(FMOD_CHANNEL_FREE, stream1, false, &channel);
	errchk(result);

	result = channel->getFrequency(&defreq);
	errchk(result);

	printf("\n Default freq is: %.2f\n\n", defreq);

	result = pitch->setParameter(FMOD_DSP_PITCHSHIFT_FFTSIZE, 4096);
	errchk(result);

	result = system->addDSP(pitch,0);
	errchk(result);

	result = pitch->setActive(active);
	errchk(result);

	while(!done)
	{
		printf("\rEnter frequency: ");
		scanf(" %f", &spfreq);

		if(spfreq > 0)
		{
			result = pitch->remove();
			errchk(result);

			result = channel->setFrequency(spfreq);
			errchk(result);

			result = pitch->setParameter(FMOD_DSP_PITCHSHIFT_PITCH, (defreq/spfreq));
			errchk(result);

			result = system->addDSP(pitch, 0);
			errchk(result);

			result = pitch->getParameter(FMOD_DSP_PITCHSHIFT_PITCH, &pit, 0, 0);
			errchk(result);

			printf("\nCurrent Pitch: %f\n", pit);
		}
		else
			done = true;
	}
	result = stream1->release();
	errchk(result);
	result = system->close();
	errchk(result);
	result = system->release();
	errchk(result);

	return 0;
}

Direction Change monitor in C++ using SDL

This is the first step towards tempo control using a cursor for my music controller project. The example shows a message whenever the direction of Y has changed while the mouse is moving and displays how long it has taken in milliseconds to change.

The code displays a simple SDL window. The window waits for a mouse motion event and displays a message to the console when Y has changed direction containing what direction it is currently going and how long it took the previous change in direction to occur. Waiting for an event proved to be more efficient than polling the event.

This logic will be the user input for controlling the tempo in the final project but adapted to work with Kinect.

#include "stdafx.h"

int _tmain(int argc, _TCHAR* argv[])
{
    float x,y;
    int direction = -1;
    bool done = false;
    Uint32 now = 0, prev = 0, interval =0;

    SDL_Surface *screen = NULL;
    SDL_Event event;

    SDL_Init(SDL_INIT_EVERYTHING);

    screen = SDL_SetVideoMode( 500, 500, 32, SDL_SWSURFACE );
    SDL_Flip( screen );

    while((!done) && (SDL_WaitEvent(&event)))
    {
        switch(event.type)
        {
            case SDL_MOUSEMOTION:
                y = event.motion.yrel;
 
                if(direction == -1)
                {
                    if(y < -1) 
                    { 
                        direction = 1; 
                        now = SDL_GetTicks(); 
                        interval = now - prev; 
                        prev = now; 
                        printf("Direction changed to 1. Y is decreasing(upscreen), 
                                    time interval %i\n",               
                                    interval); 
                    } 
                } else 
                { 
                    if(y > 1)
                    {
                        direction = -1;
                        now = SDL_GetTicks();
                        interval = now - prev;
                        prev = now;
                        printf("Direction changed to -1. Y is increasing(downscreen), 
                                      time interval %i\n", interval);
                    }
                }
                break;
            case SDL_QUIT:
                done = true;
                break;
        }
    }
    return 0;
}

Kinect Progress and Project Poster

I was able to get my hands on a Kinect for Windows. I’ve started doing tutorials for basic tracking such as sensor initialization and depth tracking. I practiced these techniques by following these tutorials:

http://channel9.msdn.com/Series/KinectQuickstart

For our presentation, at project poster is required to provide a overview of the project, so I spent the weekend doing a design of what I hope the final application should look like.

Controlling Looping using Volume

In this application, I aimed to demonstrate a ‘game’ scenario where the user can only progress to the next section of the track by fulfilling the required conditions. For this example, I decided to keep it simple by using volume as a control. My overall idea is to create may variables such as filtering certain frequencies or activating effects, this is just to get the looping functionality down.

Also the ‘magic numbers’ in the code specify the loop start and end time in milliseconds. These were just estimates from Reaper, in the future I intend to have a text file or similar to store these numbers since they are different for each track.

#include "stdafx.h"
#include "fmod.hpp"
#include "fmod_errors.h"
#include <windows.h>
#include <stdio.h>
#include <conio.h>
void errchk(FMOD_RESULT);
int _tmain(int argc, _TCHAR* argv[])
{
 FMOD::System *system;
 FMOD::Channel *channel = 0;
 FMOD::Sound *stream1;

int key = 0, section = 1;
 unsigned int version;

bool next = false;

float vol = 0.5;

FMOD_RESULT result;

result = FMOD::System_Create(&system);
 errchk(result);

result = system->getVersion(&version);
 errchk(result);

if (version < FMOD_VERSION)
 {
 printf("Error! You are using an old version of FMOD %08x. This program requires %08x\n", version, FMOD_VERSION);
 return 0;
 }

result = system->init(1, FMOD_INIT_NORMAL, 0);
 errchk(result);

result = system->createStream("C:/trackStreams/Balcony.wav", FMOD_HARDWARE | FMOD_LOOP_NORMAL | FMOD_2D, 0, &stream1);
 errchk(result);

result = system->playSound(FMOD_CHANNEL_FREE, stream1, false, &channel);
 errchk(result);

result = channel->setVolume(vol);
 errchk(result);

result = channel->setLoopPoints(14, FMOD_TIMEUNIT_MS, 8395, FMOD_TIMEUNIT_MS);
 errchk(result);

do
 {
 if(_kbhit())
 {
 key = _getch();

switch(key)
 {
 case 'a':
 if(vol < 1.0)
 {
 vol += 0.1f;

result = channel->setVolume(vol);
 errchk(result);
 break;
 }
 case 's':
 if(vol > 0.0)
 {
 vol -= 0.1f;

result = channel->setVolume(vol);
 errchk(result);
 break;
 }
 }
 }

switch(section)
 {
 case 1:
 {
 if(vol < 0.4)
 {
 channel->setMode(FMOD_LOOP_OFF);
 unsigned int pos;

channel->getPosition(&pos, FMOD_TIMEUNIT_MS);
 if(pos > 8300)
 {
 printf("\n\nSection 2 started:%d\n", pos);
 result = channel->setLoopPoints(8395, FMOD_TIMEUNIT_MS, 13406, FMOD_TIMEUNIT_MS);
 errchk(result);
 section++;
 }
 }
 else
 {
 channel->setMode(FMOD_LOOP_NORMAL);
 }
 break;
 }
 case 2:
 {
 if(!next)
 {
 if(vol >= 0.7)
 {
 channel->setMode(FMOD_LOOP_OFF);
 unsigned int pos;

channel->getPosition(&pos, FMOD_TIMEUNIT_MS);
 if(pos > 13350)
 {
 printf("\n\nSection 3 started:%d\n", pos);
 section++;
 next = true;
 }
 }
 else
 {
 channel->setMode(FMOD_LOOP_NORMAL);
 }
 }
 break;
 }
 }

system->update();

{
 float volume;

result = channel->getVolume(&volume);
 errchk(result);

printf("Channel volume: %.f\r", volume*100);
 }
 Sleep(10);
 } while (key != 27);

result = stream1->release();
 errchk(result);
 result = system->close();
 errchk(result);
 result = system->release();
 errchk(result);

return 0;
}

void errchk(FMOD_RESULT r)
{
 if (r != FMOD_OK)
 {
 printf("FMOD error! (%d) %s\n", r, FMOD_ErrorString(r));
 //exit(-1);
 }
}

Looking at getting some screen captures done of these examples to explain them better. And somewhere better to store code….