Dear Diary,
Today it finally happened :) Kinda.
So as of last Friday night's hacking, if you choose "test recording", talkLock will record audio and post it (in short chunks) over and over until you choose "Big Red Button". If you choose "get audio", it will get audio from the server and play it over and over until you choose "Big Red Button".
But I was the only one with a java phone to test it!
So I wrote a little shell script on my Mac using a copy of ffplay that I compiled with amr-nb support. I used to curl to fetch audio from the server and feed it to ffplay. Unfortunately ffplay doesn't exit when it finishes playing a sound, so I backgrounded ffplay. This kind of fork-bombed my Mac :o So I had to make my script look for pids for ffplay processes and kill them after playing. This was slow and insane, but I was able to talk into my phone after choosing "test recording", and start my script on the Mac, and see the ffplay visualization of what I said, as well as hear it :) It was choppy, or should I say there were dropouts in the audio of up around a second. But it was doing the thing!
Muahahahahahaha!!!
The horrible and ugly thing I finally figured out from some professor's website, was a trick using a new class that implements Runnable. So I declared a quit() method, and a run() method that did the whole record and POST thing. Then in my main midlet code, I instantiated my Runnable as puttit. The trick is that you pass your Runnable as an argument to a new thread, like "Thread putting = new Thread(puttit);". Then I could call putting.start() to record and POST, and puttit.quit() to make the run() method stop. Whew.
So I didn't have to do anything with mutexes or synchronized or wait or notify. I'm glad of that, especially after reading Roedy Green say to avoid that paradigm.
The most fun though, was saying in a long drawn out monotone, "feeeeeeeddddbbaaaaaacccckkkk" into the phone, which was very close to the Mac. This of course caused a feedback loop as the Mac sound output went back into the phone again, getting more and more distorted and tinny and strange with every iteration :) Wonderful, great fun!
So the performance isn't all there yet, and the code is probably very inefficient, and using "Big Red Button" stops the presses, but the program is unstable afterwards... More hackin'.
Monday, October 27, 2008
Monday, October 20, 2008
more research, synchronized wait and notify
Well, I thought that I understood how to write MIDlets, until I started putting together the pieces of talkLock. Turns out that I have more research to do.
The problem is that in a Mobile Java program, or MIDlet, you don't really have a main(). You get a startApp() method, but it really only gets run once. And your constructor, which in talkLock is named talkLock() (and in a MIDlet called Test it would have to be named test()), of course only gets run to start up your program. So the only part of the code that can get run over and over again, and do things that you would normally do in your main(), are the Commands. But Commands require user input on their menu button or keyboard or touchscreen. So what do you do with code that does something that uses a lot of cpu time, like recording audio and http POSTing it to a server?
The answer is you put that code in a separate thread. But how do you tell that thread to do it's job over and over again, if a condition is met? How do you tell it when it's time to do it's work again? It looks like this is handled by synchronize, wait, notify.
The sample code and articles I have been reading on synchronize, wait, and notify explain how to do this in a MIDlet. But what does it mean to synchronize? What does notify really do? These are java-isms.
So I am having to learn another new paradigm, this synchronize, wait, notify architecture. It's pretty limited really, I will have to write some test code to really analyze the behavior.
Here is what I understand so far: That objects are synchronized. The "synchronized" keyword means that any objects in my code that define as "synchronized" share a spinlock or mutex. Or in MVC terminology (another paradigm that I am learning to apply to this MIDlet model), these objects are Monitored.
So these objects share a spinlock. In the code for these objects, I have them check the state of some global or globals. If they are supposed to be doing something, as defined in this global or globals, then they call wait(). This tells them that they need to hear from the other objects that they share a spinlock with that it is time for them to try to grab the lock and do their work. After the wait() call, they need to check globals again, and do their work.
So if these objects are in separate threads, they can just hang out in memory, and wait for their globals to be set appropriately, and then run off and do some cpu-intensive job. The same behavior as if they were in a main(), but split off into separate threads, and using the synchronized, wait, notify concept.
How do they know it's time for them to do their work? When a notify() is sent (if you have more than two objects on the spinlock, you notifyAll()), then the synchronized objects check their globals. So if you carefully sequence events so that only in certain situations will the objects be triggered to run, then whenever a notifyAll() happens, you know what state the code will be in, and what event will be triggered, by assigning a new value to a global before you notifyAll().
By working this way you can force your code to act like it would in a normal program with a main(). I guess the idea is that your MIDlet will always be responsive to user input, and your mutexed objects will run in separate threads to make sure that main program loop (really just the system listening for user input) will always be available.
At least that's the picture I've painted for myself so far. I think I need some time to absorb these concepts, and maybe depress a little before I write my first test code. I can see a lot of frustrating debugging to get this model right. It's very difficult for me to move away from the 1980s batch programming mindset. As you might have noticed, I still look at this event-driven model from a mindset where I'm trying to make it act like a batch program with a big state machine in it. Old dogs are slow to learn new tricks, if they do.
I can understand why this model of programming is popular, you get a lot of your gui handling stuff for free this way. But I don't think that every problem is best solved with a MVC model.
Actually I think that this will let me come up with a completely new design for talkLock architecturally. Maybe I'll just put all of my code into one synchronized object that runs in a separate thread, and that code will be a big state machine :) It will be my main(), and will just be operated on by the CommandListener setting certain globals and pinging with notify(). Then I can learn this new paradigm and use it without really changing the way I write that much :)
I'll do more research, and read some source code: maybe that's really how people use it anyway :)
The problem is that in a Mobile Java program, or MIDlet, you don't really have a main(). You get a startApp() method, but it really only gets run once. And your constructor, which in talkLock is named talkLock() (and in a MIDlet called Test it would have to be named test()), of course only gets run to start up your program. So the only part of the code that can get run over and over again, and do things that you would normally do in your main(), are the Commands. But Commands require user input on their menu button or keyboard or touchscreen. So what do you do with code that does something that uses a lot of cpu time, like recording audio and http POSTing it to a server?
The answer is you put that code in a separate thread. But how do you tell that thread to do it's job over and over again, if a condition is met? How do you tell it when it's time to do it's work again? It looks like this is handled by synchronize, wait, notify.
The sample code and articles I have been reading on synchronize, wait, and notify explain how to do this in a MIDlet. But what does it mean to synchronize? What does notify really do? These are java-isms.
So I am having to learn another new paradigm, this synchronize, wait, notify architecture. It's pretty limited really, I will have to write some test code to really analyze the behavior.
Here is what I understand so far: That objects are synchronized. The "synchronized" keyword means that any objects in my code that define as "synchronized" share a spinlock or mutex. Or in MVC terminology (another paradigm that I am learning to apply to this MIDlet model), these objects are Monitored.
So these objects share a spinlock. In the code for these objects, I have them check the state of some global or globals. If they are supposed to be doing something, as defined in this global or globals, then they call wait(). This tells them that they need to hear from the other objects that they share a spinlock with that it is time for them to try to grab the lock and do their work. After the wait() call, they need to check globals again, and do their work.
So if these objects are in separate threads, they can just hang out in memory, and wait for their globals to be set appropriately, and then run off and do some cpu-intensive job. The same behavior as if they were in a main(), but split off into separate threads, and using the synchronized, wait, notify concept.
How do they know it's time for them to do their work? When a notify() is sent (if you have more than two objects on the spinlock, you notifyAll()), then the synchronized objects check their globals. So if you carefully sequence events so that only in certain situations will the objects be triggered to run, then whenever a notifyAll() happens, you know what state the code will be in, and what event will be triggered, by assigning a new value to a global before you notifyAll().
By working this way you can force your code to act like it would in a normal program with a main(). I guess the idea is that your MIDlet will always be responsive to user input, and your mutexed objects will run in separate threads to make sure that main program loop (really just the system listening for user input) will always be available.
At least that's the picture I've painted for myself so far. I think I need some time to absorb these concepts, and maybe depress a little before I write my first test code. I can see a lot of frustrating debugging to get this model right. It's very difficult for me to move away from the 1980s batch programming mindset. As you might have noticed, I still look at this event-driven model from a mindset where I'm trying to make it act like a batch program with a big state machine in it. Old dogs are slow to learn new tricks, if they do.
I can understand why this model of programming is popular, you get a lot of your gui handling stuff for free this way. But I don't think that every problem is best solved with a MVC model.
Actually I think that this will let me come up with a completely new design for talkLock architecturally. Maybe I'll just put all of my code into one synchronized object that runs in a separate thread, and that code will be a big state machine :) It will be my main(), and will just be operated on by the CommandListener setting certain globals and pinging with notify(). Then I can learn this new paradigm and use it without really changing the way I write that much :)
I'll do more research, and read some source code: maybe that's really how people use it anyway :)
Monday, October 13, 2008
mobile java, threads and globals, example source code
So, I started with a clean project in Netbeans, talkLock_beta. The idea is that now that most of my R&D work is done, I should try to design talkLock rather than just hack stuff together.
I have done so, and made a midlet that does a simple test. I increment an integer variable in a thread, a global integer variable for my midlet. The midlet knows that the integer was incremented in the thread. So I don't have to do any crazy message passing things with threads, just use globals. Yay!
I could clean up the code and call it threadsandglobals or something, but for now, here it is.
// includes
import javax.microedition.midlet.*;
import javax.microedition.lcdui.*;
import javax.microedition.lcdui.List.*;
import javax.microedition.media.*;
import javax.microedition.media.control.*;
import java.io.*;
import javax.microedition.rms.*;
import javax.microedition.rms.RecordEnumeration.*;
import javax.microedition.io.*;
// our midlet
public class talkLock_beta extends MIDlet implements CommandListener {
// Commands
private Command runthreads;
private Command getvalues;
private Command resetvalues;
private Command exit;
//Forms
Form screenForm;
// Strings
// Ints
int getchunknum;
int postchunknum;
// Displays
Display thescreen;
// our void main
public talkLock_beta(){
// construct our main screen, a Form
screenForm = new Form("talkLock");
// construct our Commands for the CommandListener
exit = new Command("exit", Command.EXIT, 1);
runthreads = new Command("run threads", Command.SCREEN, 0);
getvalues = new Command("get values", Command.SCREEN, 0);
resetvalues = new Command("reset values", Command.SCREEN, 0);
// add our Commands to the Form
screenForm.addCommand(exit);
screenForm.addCommand(runthreads);
screenForm.addCommand(getvalues);
screenForm.addCommand(resetvalues);
// ping the user
screenForm.append("midlet is alive");
// initialize our counters
postchunknum = 1;
getchunknum = 1;
} // end of our void main
// our runthreads method
public void runthreads(){
// the GET thread
Thread getthread = new Thread(){
public void run(){
++getchunknum;
} // end of run
}; // end of getthread
// the POST thread
Thread postthread = new Thread(){
public void run(){
++postchunknum;
} // end of run
}; // end of getthread
// see how they run
postthread.start();
getthread.start();
} // end of runthreads method
// midlet lifecycle destroyApp method
public void destroyApp (boolean unconditional) { }
// midlet lifecycle pauseApp method
public void pauseApp() { }
// midlet lifecycle startApp method
public void startApp() {
// initialize our Display
thescreen = Display.getDisplay(this);
// set our Form as the current Display
thescreen.setCurrent(screenForm);
// flag the CommandListener to listen to our Display object
screenForm.setCommandListener(this);
}
// we have to define our methods for the CommandListener to call
public void commandAction (Command c, Displayable s) {
// our exit command
if(c == exit){
destroyApp(false); notifyDestroyed();
} // end of exit
// our runthreads command
else if (c == runthreads){
runthreads();
} // end of runthreads
// our getvalues command
else if (c == getvalues){
screenForm.append("" + getchunknum);
screenForm.append("" + postchunknum);
} // end of getvalues
// our resetvalues command
else if (c == resetvalues){
getchunknum = 0;
postchunknum = 0;
} // end of resetvalues
} // end of commandAction
} // end of midlet
I have done so, and made a midlet that does a simple test. I increment an integer variable in a thread, a global integer variable for my midlet. The midlet knows that the integer was incremented in the thread. So I don't have to do any crazy message passing things with threads, just use globals. Yay!
I could clean up the code and call it threadsandglobals or something, but for now, here it is.
// includes
import javax.microedition.midlet.*;
import javax.microedition.lcdui.*;
import javax.microedition.lcdui.List.*;
import javax.microedition.media.*;
import javax.microedition.media.control.*;
import java.io.*;
import javax.microedition.rms.*;
import javax.microedition.rms.RecordEnumeration.*;
import javax.microedition.io.*;
// our midlet
public class talkLock_beta extends MIDlet implements CommandListener {
// Commands
private Command runthreads;
private Command getvalues;
private Command resetvalues;
private Command exit;
//Forms
Form screenForm;
// Strings
// Ints
int getchunknum;
int postchunknum;
// Displays
Display thescreen;
// our void main
public talkLock_beta(){
// construct our main screen, a Form
screenForm = new Form("talkLock");
// construct our Commands for the CommandListener
exit = new Command("exit", Command.EXIT, 1);
runthreads = new Command("run threads", Command.SCREEN, 0);
getvalues = new Command("get values", Command.SCREEN, 0);
resetvalues = new Command("reset values", Command.SCREEN, 0);
// add our Commands to the Form
screenForm.addCommand(exit);
screenForm.addCommand(runthreads);
screenForm.addCommand(getvalues);
screenForm.addCommand(resetvalues);
// ping the user
screenForm.append("midlet is alive");
// initialize our counters
postchunknum = 1;
getchunknum = 1;
} // end of our void main
// our runthreads method
public void runthreads(){
// the GET thread
Thread getthread = new Thread(){
public void run(){
++getchunknum;
} // end of run
}; // end of getthread
// the POST thread
Thread postthread = new Thread(){
public void run(){
++postchunknum;
} // end of run
}; // end of getthread
// see how they run
postthread.start();
getthread.start();
} // end of runthreads method
// midlet lifecycle destroyApp method
public void destroyApp (boolean unconditional) { }
// midlet lifecycle pauseApp method
public void pauseApp() { }
// midlet lifecycle startApp method
public void startApp() {
// initialize our Display
thescreen = Display.getDisplay(this);
// set our Form as the current Display
thescreen.setCurrent(screenForm);
// flag the CommandListener to listen to our Display object
screenForm.setCommandListener(this);
}
// we have to define our methods for the CommandListener to call
public void commandAction (Command c, Displayable s) {
// our exit command
if(c == exit){
destroyApp(false); notifyDestroyed();
} // end of exit
// our runthreads command
else if (c == runthreads){
runthreads();
} // end of runthreads
// our getvalues command
else if (c == getvalues){
screenForm.append("" + getchunknum);
screenForm.append("" + postchunknum);
} // end of getvalues
// our resetvalues command
else if (c == resetvalues){
getchunknum = 0;
postchunknum = 0;
} // end of resetvalues
} // end of commandAction
} // end of midlet
Wednesday, October 8, 2008
RecordStore works
Well, I figured out the problem with retrieving records from RecordStore. Whatever call I was using to get the byte[] of data was totally wrong. I must have read the wrong API doc or something.
Once I figured that out, I realized that the myrecordstore.getRecord() method returns your data, which I was throwing away. I assigned a variable to collect the data to each call. That helps :)
Then I was returning a StringBuffer from my getrecords() method that did not contain the data from my records, but a reference hash to the object containing my data. I found out that with String, if you run a constructor feeding your byte[] data to it, it will dereference it for you. Like: String stufffromrecordstore = new String(bytearrayreferencefromgetrecord). The "new String()" part runs the constructor and does the dereference magic. Cool huh?
So now in the emulator I get a nice Alert() that prints the stuff from the RecordStore. Yay!
Now how am I going to make all this work together? Hmm... The challenge is to design thigns and *not* prematurely optimize.
Once I figured that out, I realized that the myrecordstore.getRecord() method returns your data, which I was throwing away. I assigned a variable to collect the data to each call. That helps :)
Then I was returning a StringBuffer from my getrecords() method that did not contain the data from my records, but a reference hash to the object containing my data. I found out that with String, if you run a constructor feeding your byte[] data to it, it will dereference it for you. Like: String stufffromrecordstore = new String(bytearrayreferencefromgetrecord). The "new String()" part runs the constructor and does the dereference magic. Cool huh?
So now in the emulator I get a nice Alert() that prints the stuff from the RecordStore. Yay!
Now how am I going to make all this work together? Hmm... The challenge is to design thigns and *not* prematurely optimize.
Monday, October 6, 2008
pre-alpha_006 - we have gathered most of the pieces
So we have most of the pieces to make talkLock work now. On Blackberry, we can record audio, POST it to the webserver, which then fixes the audio data and base64 decodes it. For some reason the audio data (base64 encoded) loses "+" signs, they are replaced with spaces. Not sure if Apache or PHP does this. But anyway, talk.php (the webserver script side) fixes that. I can then GET the audio and it plays back :)
This is a BIG deal. This means that on Blackberry, I have everything I need to do communication.
I have fixed the code that allowed the audio handling to work on the LG CU500, which I found out is a Qualcomm based phone. However, the Blackberry cannot play audio from the LG, and the LG cannot play audio from the LG.
I think that to troubleshoot this I will take audio from the Blackberry, and audio from the LG, and feed them to ffmpeg to see what the difference is. I hate to make talkLock depend on ffmpeg compiled with amrnb support. That would make things a lot more complicated for folks to install the server side talkLock component. It makes me want to look at perhaps a different programming language for the server side. If Java knows about amrnb for example, then I could just use Java. We'll see.
Also ffmpeg doesn't have a lot of options to convert from amrnb to wav and back, but it has very good ogg support. So I might have to convert from amrnb to ogg to wav, and make all audio that is fetched from the server wav. That is a big hammer for a small nail though.
But we have all of the pieces! Time to hack up a demo option to do two-way communication on Blackberry ;)
This is a BIG deal. This means that on Blackberry, I have everything I need to do communication.
I have fixed the code that allowed the audio handling to work on the LG CU500, which I found out is a Qualcomm based phone. However, the Blackberry cannot play audio from the LG, and the LG cannot play audio from the LG.
I think that to troubleshoot this I will take audio from the Blackberry, and audio from the LG, and feed them to ffmpeg to see what the difference is. I hate to make talkLock depend on ffmpeg compiled with amrnb support. That would make things a lot more complicated for folks to install the server side talkLock component. It makes me want to look at perhaps a different programming language for the server side. If Java knows about amrnb for example, then I could just use Java. We'll see.
Also ffmpeg doesn't have a lot of options to convert from amrnb to wav and back, but it has very good ogg support. So I might have to convert from amrnb to ogg to wav, and make all audio that is fetched from the server wav. That is a big hammer for a small nail though.
But we have all of the pieces! Time to hack up a demo option to do two-way communication on Blackberry ;)
Subscribe to:
Posts (Atom)
