Wednesday, September 19, 2012

Talking Pets: Development Pet for Android - Part 2


Talking Pets: Development Pet for Android - Part 2
The second part of a series of articles on the development of a talking pet for Android. In this article we will discuss how to play the recorded data. We create a working application that will continuously record and play voice.



What was used


An AudioTrack instance can operate under two modes: static or streaming.
In Streaming mode, the application writes a continuous stream of data to the AudioTrack, using one of the write() methods. These are blocking and return when the data has been transferred from the Java layer to the native layer and queued for playback.
The static mode should be chosen when dealing with short sounds that fit in memory and that need to be played with the smallest latency possible. The static mode will therefore be preferred for UI and game sounds that are played often, and with the smallest overhead possible.
Upon creation, an AudioTrack object initializes its associated audio buffer. The size of this buffer, specified during the construction, determines how long an AudioTrack can play before running out of data.

Practice

  • Voice playback
Basic constants and attributes of the class:
 private static int FREQUENCY   = 40000;          // recording frequency
 private static final int CHANNEL  = AudioFormat.CHANNEL_CONFIGURATION_STEREO; // mono or stereo channel
 private static final int ENCODING  = AudioFormat.ENCODING_PCM_16BIT;   // data encoding
 
 // sample rates
 private static final int[] mSampleRates = new int[] { 44100, 22050, 16000, 11025, 8000 };
 
 private boolean isPlay = false; //check, if the sound is played
 
 // data for playback
 private int size;
 private byte[] array;
 
 private int bufSize = AudioTrack.getMinBufferSize(FREQUENCY, CHANNEL, ENCODING); // buffer size
 
 private Handler handler;// to send messages to the application UI thread
 
 // Used to play back audio data
 private AudioTrack audioTrack = new AudioTrack(AudioManager.STREAM_SYSTEM,
   FREQUENCY, CHANNEL, ENCODING, bufSize, AudioTrack.MODE_STREAM);

You must first initialize AudioTrack:
public Playback(Handler handler) {
  DebugLog.i(TAG, "Playback()");
  this.handler = handler;
  this.audioTrack = findAudioTrack();
 }

 /**
  * Used to search for possible AudioTrack to the settings for the device
  */
 public AudioTrack findAudioTrack() {
  DebugLog.i(TAG, "findAudioTrack()");
  // Search the minimum buffer size
  this.bufSize = AudioTrack.getMinBufferSize(FREQUENCY, CHANNEL, ENCODING);
  if (this.bufSize != AudioRecord.ERROR_BAD_VALUE) {
   
   // In this case, you will create AudioTrack with a frequency greater than the record.
   // In this way, we obtain a "squeaky" voice of a character.
   
   AudioTrack at = new AudioTrack(AudioManager.STREAM_SYSTEM,
     FREQUENCY, CHANNEL, ENCODING, bufSize,
     AudioTrack.MODE_STREAM);

   if (at.getState() == AudioRecord.STATE_INITIALIZED)
    return at;
  }   

  // If you create the AudioTrack can not, then try to create it with a frequency greater than exactly 2 times than during the recording
  FREQUENCY = 2 * Record.FREQUENCY;
  this.bufSize = AudioTrack.getMinBufferSize(FREQUENCY, CHANNEL, ENCODING);
  if (this.bufSize != AudioRecord.ERROR_BAD_VALUE) {
   AudioTrack at = new AudioTrack(AudioManager.STREAM_SYSTEM,
     FREQUENCY, CHANNEL, ENCODING, this.bufSize,
     AudioTrack.MODE_STREAM);

   if (at.getState() == AudioRecord.STATE_INITIALIZED)
    return at;
  }
  // If all attempts fail, try to create at least some AudioTrack
  for (int rate : mSampleRates) {
   try {
    this.bufSize = AudioTrack.getMinBufferSize(rate, CHANNEL, ENCODING);

    if (this.bufSize != AudioRecord.ERROR_BAD_VALUE) {
     FREQUENCY = rate;
     AudioTrack at = new AudioTrack(AudioManager.STREAM_SYSTEM,
       FREQUENCY, CHANNEL, ENCODING, this.bufSize,
       AudioTrack.MODE_STREAM);

     if (at.getState() == AudioTrack.STATE_INITIALIZED)
      return at;
    }
   } catch (Exception e) {
    e.printStackTrace();
   }
  }
  return null;
 }
The basic method is to run, in which we record the audio data in AudioTrack and play them.
public void run() {
  try {
   DebugLog.i(TAG, "run()");
   
   isPlay = true;  
   // voice reproduction
   audioTrack.play();

   audioTrack.write(array, 0, size);
   audioTrack.stop();
   audioTrack.release();
   if (isPlay)
   {
    // start a voice recording
    handler.sendEmptyMessage(Constants.MSG_RECORD);
   }
   isPlay = false;

  } catch (Exception e) {
   isPlay = false;
   e.printStackTrace();
  }
 }
  • The work of the algorithm
For correct operation the algorithm I use to send messages using Handler.
There are four states:
  • Record data;
  • Voice recording, animation;
  • Playback of data;
  • Play, animation.
 // processing messages from objects
 private Handler handler = new Handler() {
  public void handleMessage(Message msg) {
   DebugLog.i(TAG, "msg.what = " + msg.what);
   
   if (isPause)
    return;
   
   // display status
   if (msg.what >= 0 && msg.what < array_condition.length)
    txt_condition.setText(array_condition[msg.what]);
   
   switch (msg.what) {
    case Constants.MSG_RECORD:  // voice recording
     if ((record != null) && (!record.isRec())) 
      record.startRecord();    
    break;
    case Constants.MSG_PLAYBACK: // voice reproduction
     playback = new Playback(handler);
     int size = msg.getData().getInt(Constants.DATA_SIZE);
     byte[] array = msg.getData().getByteArray(Constants.DATA_ARRAY);
     playback.setData(size, array);
     playback.start();    
     break;
    case Constants.MSG_LISTEN:   // character listens
     break;
    }
  }
 };
  • Main Activity
Now let's talk about how to initialize, start, and stop these flows.
public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.main);
        DebugLog.i(TAG, "onCreate()");
        
        this.txt_condition = (TextView)findViewById(R.id.txt_condition);
        this.array_condition = getResources().getStringArray(R.array.array_condition);
        
        PowerManager pm = (PowerManager) getSystemService(Context.POWER_SERVICE);
  this.wl = pm.newWakeLock(PowerManager.SCREEN_BRIGHT_WAKE_LOCK
    | PowerManager.ACQUIRE_CAUSES_WAKEUP, "TAG");
  
   
    }
    
    protected void onPause() {
  super.onPause();
  DebugLog.i(TAG, "onPause()");
  this.wl.release();
  isPause = true;
  
  if ((this.playback != null) && (this.playback.isPlay())) {
   this.playback.stopPlay();
  }
  // stop recording
  if (this.record != null)
  {
   this.record.stopRecord();
   this.record.close();
  }
 }
    
    protected void onResume(){
     super.onResume();
     DebugLog.i(TAG, "onResume()");
     this.wl.acquire();
     isPause = false;     
     // runnable threads for voice recording
     this.record = new Record(handler);   
     this.record.start();
  handler.sendEmptyMessage(Constants.MSG_RECORD);
    }

Links

  • The source codes of this project can be downloaded here: zip