how to detect a music note from stave

[LisaW]

---

@ShazilM wrote:

Hi,

No worries. Hope it helps. Yeah array sounds fine, however I meant to ask how are you "Physically" communicating between the robots? Are they connected via ethernet for example?

---

They're not connected physically, we only need the first "robot" to detect the music notes and to send that information to the program that controls the second robot. Or isn't that possible to do with only a program?

[LisaW]

---

@Alain_S wrote:

Now I'm a bit confused 😞

In your first post you wrote that the "playing" robot is working.
Reading this, one can assume that you give this robot some input (data) to play something, the format of the data is known, right?

In mesage 8 you wrote that you aren't sure how to send the data from the "reading" robot to the "playing" robot?

So, like I said, I'm a bit confused ?

You also wrote that real time reading/playing is not required, right?

So you can put the data to play in a standard text file.

Making a file manually allows you to debug the "playing" robot.

Are both robots controlled by one and the same computer or not?

You also wrote : "The idea was to build some sort of robot with a webcam that moves over the music sheet, "

Why not scanning a whole music sheet a once and "translate" it into a file that can be played by the "playing" robot?

The word "moves" scares me a bit 😞

Do you have a motor, a motion controller, what a the end of the staff, how to ensure parallel travel over the staff, ... ... ?

---

I apologize for the confusion, we're not 100% sure yet how we're gonna do this.

The program for controlling the piano playing robot is not completely done yet, at the moment we need to manually enter the notes we want the robot to play.

Our first plan was to use text files to send the date to the playing robot but our teacher said that would be too easy so that's why we came up with this idea.

We were inspired by a video on YouTube (https://www.youtube.com/watch?v=E4BTUJE1r8w) and since it is the first time we work with LabVIEW and NI Vision Assistant, we have no idea what possibilities we have so we thought let's try it this way.

So it is possible to scan a full music sheet and detect the music notes one by one?

[Alain_S]

What is shown on the movie is not sheet music to me, more toilet paper with some notes on it 😞 🙂

I think you should first define how far you want to go with this... ...

1) Is the "playing" robot playing 1 or 2 hands?

2) You said you want to play "easy" music but even with "easy" music a lot of things can appear on the music sheet.

3) Do you want to use real sheet music or will you make your own "toilet paper" sheet like on youtube?

I took a picture of a music sheet with a standard camera.

Load that image into VisionAssitant and made the attached script

That script uses OCR to recognize some notes on the image

I "train" each of these notes beforehand, that is the "trial-001.txt" file

That means that you have to teach each individual note.

Pleae note that the script doesn't take care of left and right hand staff!

So the same note on the staff return the same note out of the script although it's a different note on the keyboard!!

Like I said, almost impossible unless you have lots of lots of lots of lots of free time... ...

[NIquist]

I suggest an intermediate step and use already existing software to convert the score to a MIDI file.  

https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=score%20to%20midi

http://en.wikipedia.org/wiki/MIDI

MIDI is a relatively simple format from which you can extract the note, its velocity and its length.  It's better to leverage that than try to roll your own converter which, as has been mentioned, would be a great effort.

[billko]

---

@NIquist wrote:

I suggest an intermediate step and use already existing software to convert the score to a MIDI file.  

https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=score%20to%20midi

http://en.wikipedia.org/wiki/MIDI

MIDI is a relatively simple format from which you can extract the note, its velocity and its length.  It's better to leverage that than try to roll your own converter which, as has been mentioned, would be a great effort.

---

THAT is a great idea!!!

[LisaW]

---

@NIquist wrote:

I suggest an intermediate step and use already existing software to convert the score to a MIDI file.  

https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=score%20to%20midi

http://en.wikipedia.org/wiki/MIDI

MIDI is a relatively simple format from which you can extract the note, its velocity and its length.  It's better to leverage that than try to roll your own converter which, as has been mentioned, would be a great effort.

---

Thank you, I will try to work with this!

[Yamaeda]

Oh yes! Going the MIDI route open up several possibilities, you can easily find and download lots of midi files for testing purposes and since most (all?) computers can play midi through the sound card you can test your Vision reading directly on the PC before sending it to the robot.

/Y