Speaklight Project

Here is my SpeakLight project, which uses Android speech recognition to control a lamp's color.

The SpeakLight project has two components, an Android app (because iPhones are a closed system) and a remote Arduino-controlled light source.  When you say to the app, for example, "aquamarine", the light and the screen change to that color.  The SpeakLight app also recognizes "brighter", "dimmer", "rainbow", and a few other commands.

 SpeakLight app screenshots 1-2-3!

The app has a list of 3,214 color names with their RGB color values.  It listens for a spoken color name and sends that color’s value through a Bluetooth connection to the light.  The light source was initially a simple tri-color LED kit, then a brighter I²C-controlled LED kit, and is now a 75 watt DMX-controlled theater light.

This is the first version of the Arduino lamp and its color pattern on the ceiling -- as you can see, the aquamarine color is there but not uniform. 

This is the second SpeakLight output device, a BlinkM MaxM, the brightest LED kit that SparkFun had! 

Third version, with the PAR64 can light.  This is the color wash I was looking for!  

The SpeakLight app required 398 lines of Java code (and weeks of Android research), and the Arduino sketch merely 79 lines of C++.

The Android built-in onActivityResult function (see below) receives the results of all requested events.  In my case, if it’s a speech event, this function parses all recognized phrases and compares each phrase to each of the known color names.  If there’s a match, that color’s hex value is sent to the Arduino, which magically (using electrons) changes the lamp’s color.

SpeakLight onActivityResult Code

String[] colors = { "Acadia#1B1404", "Acapulco#7CB0A1", "Acid Green#A8BB19", "Acorn#6A5D1B", "Aero Blue#C9FFE5", "African Violet#B284BE", ... };

protected void onActivityResult(int requestCode, int resultCode, Intent data) {
        String result ="", firstResult = "", colorName = "";
        boolean colorFound = false;
        int iColor = 0;
        String[] cNameVal;       

        button.setText("Working...");

        if (requestCode == SPEECH_REQUEST_CODE) { 
            if (resultCode == RESULT_OK) {
                ArrayList matches = 
data.getStringArrayListExtra(RecognizerIntent.EXTRA_RESULTS);

                if (matches.size() == 0) {
                    button.setText("No matches!");
                } else {
                    // most likely matched phrase
                    firstResult = matches.get(0); 

                    for (int j = 0; j < matches.size(); j++) {
                        // each phrase in turn
                        result = matches.get(j).toLowerCase(); 
                        if (result.contains("grey")) {
                            result = result.replace("grey", "gray");
                        }
                        for (int i = 0; i < colors.length; i++) {
                            // color name [0] and value [1]
                            cNameVal = colors[i].split("#");
                            if (result.equals(cNameVal[0].toLowerCase())) {
                                colorFound = true;
                                // prepend octothorpe
                                sColor =  "#" + cNameVal[1]; 
                                iColor = parseColor(sColor);
                                colorName = cNameVal[0];
                                button.setText(colorName);
                                break;
                            }
                        }
                        if (colorFound) {
                            break;
                        }
                    }
                }
            } else {
                button.setText("Please try again");
            }
        }

        if (colorFound) {
            // send color hex value bytes to Arduino through 
            // previously-established Bluetooth connection
            ArduinoSend(sColor);    
        } else {
             button.setText("'" + firstResult + "' is not a color I know!");
             beDoop(errsound);
        }

        super.onActivityResult(requestCode, resultCode, data);
    }

XML Project

The task was to collect revision information as XML records from 477 legacy documents (most only in PDF format, some in Word doc or docx) in 44 collections over 5 releases, and also to convert all PDF files to Word files.  

The resulting historical information is up on MSDN (example).

Each document (PDF or otherwise) has a Revision Summary table containing a set of release dates, revision numbers, and revision classes, for example:

Rather than do this by hand, I wrote some PowerShell and Word Basic scripts to extract the release information from each file’s Revision Summary table and save it as XML records, one per release per file.  For each revision source directory:

1. The first PowerShell script (PDFtoDoc.ps1) goes through a given directory and opens each PDF file in Word, then saves it as a doc file.  
2. A follow-on PowerShell script (WordExtractDir.ps1) opens each new doc or legacy docx file in a given directory, and executes a Word macro (ReturnDocInfo, in ReturnDocInfo.vba) to extract and incrementally write release information records to each document's own XML file.
1. The Word macro gets the document short name (MS-XYZ) and its title, then finds the revision table and reads the last row.  The Word macro then calls a subroutine (OutputDocInfo) that writes out XML records for each entry to that document's MS-XYZ.xml file.

After all the records were extracted from all documents for all releases, I then used Bash with its sed and awk tools to post-process the 477 document XML files, also creating two summary XML files (collection structure and contents).  

I handed the set of 479 post-processed XML files to a subsequent tool writer who converted them into the format seen on MSDN.

PDFtoDoc.ps1 (PowerShell)

###
#  1. Opens each .pdf file in the given path in Word
#  2. Word saves that file in doc format
#
#  Usage: PDFtoDoc directory-path
###

$documents_path = $args[0]     # for example, c:MyProtocolPDFs\Release2014

$saveasdoc = 0                 # SaveAs format identifier

$word_app = New-Object -ComObject Word.Application

Get-ChildItem -Filter *.pdf -Path $documents_path | ForEach-Object {   

    $document = $word_app.Documents.Open($_.FullName)

    $doc_filename = "$($_.DirectoryName)\$($_.BaseName).doc"

    Write-Host $doc_filename 
    $document.SaveAs([ref] $doc_filename, [ref] $saveasdoc)

    $document.Close()
}

$word_app.Quit()

WordExtractDir.ps1 (PowerShell)

##
#  1. Starts in a directory that contains either doc or docx files
#  2. Determines which type is in this directory
#  3. Opens each doc/docx file in Word
#  4. Calls a Word macro that extracts the revision info and saves it into doc-specific XML files
#
#  Usage:  WordExtractDir directory-path
##


$dir = $args[0]

echo $dir

$i = 0
$docs = 0
$docxs = 0


$extractMacro = "Normal.NewMacros.ReturnDocInfo"

$word = New-Object -ComObject Word.Application
$word.visible = $false

echo ""

Get-ChildItem -path $dir -recurse -include "*.doc" | % {  
    $docs = $docs + 1     
}

Get-ChildItem -path $dir -recurse -include "*.docx" | % {  
    $docxs = $docxs + 1     
}

if ($docs -gt 0) { 
    $type = "doc"
    $num = $docs
} else {
    $type = "docx"    
    $num = $docxs
}


Get-ChildItem -path $dir -recurse -include "*.$type" | % {             

    $doc = $word.documents.open($_.fullname)

    $results = $word.run($extractMacro)  

    $doc.close()

    echo ([string] ($num - $i) + " - " + $_.Name)   # counts down
    $i = $i + 1
}

$word.Quit()

ReturnDocInfo.vba (Visual Basic for Applications)

Sub ReturnDocInfo()  

    Dim dirDate As String  ' Release subdirectory name
    dirDate = "2013-01-31" ' Manually set per release

    Dim bMore As Boolean   ' True if there will be another Release concatenated; False if this is the last one
    bMore = True
            
    Dim curDoc As Word.Document
                
    Dim rng As Word.Range
    Dim sDocName As String
    Dim sTitleText As String
    Dim sTitle As String
    Dim sRow As String
    Dim sDate As String
    Dim sRevDate As String
    Dim sDateCell, sChangeCell
    Dim sChange As String
    
    Dim pTitle As Paragraph
    
Debug.Print
Debug.Print "-- " + dirDate + " -- "; Now()            ' Show script starting time
    
'---------
' Get the protocol document name
'---------
    Set curDoc = ActiveDocument
    sDocName = curDoc.Name                             ' starts as "[MS-XYZ].doc"
    sDocName = Mid(sDocName, 2, Len(sDocName) - Len("].doc") - 1)
    
'---------
' Get the protocol document title
'---------
    Set rng = curDoc.Content()
    
    Set pTitle = rng.Paragraphs(1)
    sTitle = pTitle.Range.Text
    sTitleText = Mid(sTitle, 1, Len(sTitle) - 1)  ' remove one trailing character, probably CR
    sTitle = Mid(sTitleText, Len(sDocName) + 6)   ' remove leading "[MS-XYZ]: ."
    sTitle = Replace(sTitle, "", "")             ' some kind of paragraph/eol marker? only occurs on two-line titles
        
Debug.Print sDocName; " - "; sTitle
    
'---------
' Get the most recent publication date (last row in table)
'---------
    sDateCell = rng.Tables(1).Rows(rng.Tables(1).Rows.Count()).Cells(1)
    sDate = Mid(sDateCell, 1, Len(sDateCell) - 2)  ' remove two trailing characters, probably CR & LF
    sRevDate = Mid(sDate, 7, 4) + "-" + Mid(sDate, 1, 2) + "-" + Mid(sDate, 4, 2)
    
'---------
' Get the change type
'---------
    sChangeCell = rng.Tables(1).Rows(rng.Tables(1).Rows.Count()).Cells(3)
    sChange = Mid(sChangeCell, 1, Len(sChangeCell) - 2)
    If sChange = "No change" Then sChange = "None" 
    
Debug.Print sRevDate; " - "; sChange

'---------
' Write out this XML record
'---------
Call OutputDocInfo("C:\Output", dirDate, bMore, sDocName, sTitle, sRevDate, sChange)
    
End Sub

'---------------------------------------------------------------------

Sub OutputDocInfo(folderPath As String, dirDate As String, more As Boolean, docName As String, docTitle As String, docDate As String, docChange As String)

    Const sExtension = ".xml"
    
    Dim bFirstTime As Boolean
    Dim sAny As String
    
    Dim quote As String
    quote = """"
    
    Dim sBasePath As String
    sBasePath = "\\base-path\release-dir\" + dirDate
    
    Dim sIntroXML As String
    Dim sDocLoc As String
    Dim sPDFLoc As String
    
    sIntroXML = "<?xml version=" + quote + "1.0" + quote + " encoding=" + quote + "utf-8" + quote + " ?>"
    sWordLoc = "      <DocumentFile Type=" + quote + "Word" + quote + " Location="
    sPDFLoc = "      <DocumentFile Type=" + quote + "PDF" + quote + " Location="
    
    
    Dim sFileName As String
    sFileName = folderPath + "\" + docName + sExtension
    
    Set fso = CreateObject("Scripting.FileSystemObject")
    
    bFirstTime = Not fso.FileExists(sFileName)
    
    Set textStream = fso.OpenTextFile(sFileName, 8, True)   ', ForAppending, Create)
    
    If bFirstTime Then
        textStream.writeline (sIntroXML)
        sAny = "<Protocol Name=" + quote + docName + quote + " Title=" + quote + docTitle + quote + ">"
        textStream.writeline (sAny)
        textStream.writeline ("  <Releases>")
    End If
            
    sAny = "    <Release PublishDate=" + quote + docDate + quote + " ProtocolRevision=" + quote + "None" + quote + " DocumentRevision=" + quote + docChange + quote + ">"
    textStream.writeline (sAny)
    
    sAny = sPDFLoc + quote + sBasePath + "\Downloads\[" + docName + "].pdf" + quote + " />"
    textStream.writeline (sAny)
    
    sAny = sWordLoc + quote + sBasePath + "\Documents\[" + docName + "].doc" + quote + " />"
    textStream.writeline (sAny)
    
    sAny = "    </Release>"
    textStream.writeline (sAny)
    
    If Not more Then
        sAny = "  </Releases>"
        textStream.writeline (sAny)
   
        sAny = "</Protocol>"
        textStream.writeline (sAny)
    End If
   
End Sub

Unity Project

This tiny game demonstrates the primary features of a video game, where you control the player’s movements and the Unity game engine determines what happens in a collision.

The object of this game is to roll the ball into the cubes, either by tilting the device or pushing the ball with a finger. This version is geared toward an Android phone, but it can run on any device with a Unity driver.

When a cube is touched by the ball, the C# code in PlayerController.cs performs a simple game action – playing a sound (Homer Simpson saying "Woo-hoo!") and destroying that cube – see the code listing below.

The cubes have a different picture on each face, and the playing field is Easter Island.  The ball is actually white with a reddish-purple light source shining on it, and the game camera follows the ball rather than staying put over the game field. The walls are now glued down – in the first iteration, as the ball and each wall weigh 1 Unity mass unit, a ball bounce knocked that wall clean off!

PlayerController.cs

using UnityEngine;
using System.Collections;

public class PlayerController : MonoBehaviour   // this controller code is attached to the player object (in this case the ball)
{
	public float speed = 300.0f;            // overall game running speed factor
	public float touchSpeed = 1.0f;         // touch input speed factor
	public GUIText countTxt;                // display string for cubes-to-go -- GUIText variables map to Unity scene text variables
	public GUIText winTxt;                  // display string for winning!
	public int maxcount = 5;                // number of target cubes

	private int count;                      // count of cubes-to-go
	private string xaxe = "Horizontal";     // Unity-specific axis names, for example joystick axes
	private string yaxe = "Vertical";
	private Vector3 gohere;                 // next ball movement's vector
	private GameObject go;                  // each cube is a game object
	float horiz, vert;                      // incoming movement axis values

	void Start ()                           // runs once at startup
	{
		winTxt.enabled = false;		
		SetCountTxt ();

		if (Application.platform == RuntimePlatform.Android) {
			Screen.orientation = ScreenOrientation.LandscapeLeft;
			Screen.sleepTimeout = SleepTimeout.NeverSleep;
			Screen.autorotateToLandscapeLeft = true;
		}

		for (int i = 0; i < maxcount; i++) {  // instantiate cubes
			go = (GameObject)Instantiate (Resources.Load ("MyCube"), 
                             new Vector3 (Random.Range (-9.0F, 9.0F), 1.0f, Random.Range (-9.0F, 9.0F)), 
                             Quaternion.identity);
		}
	}

	void Update ()                            // runs once per game frame, such as 30fps
	{
		if (Input.GetKey (KeyCode.Escape)) {
			Application.Quit ();
		}

		// check whether there's been a moving touch on the screen; if so, move the ball correspondingly
		if (Input.touchCount > 0 && Input.GetTouch (0).phase == TouchPhase.Moved) {
			Vector2 touchDeltaPosition = Input.GetTouch (0).deltaPosition * touchSpeed;
			moveMe (touchDeltaPosition.x, touchDeltaPosition.y);
		}
	}

	void FixedUpdate ()                               // runs once per physics time tick, by default 0.02 seconds
	{
		if (SystemInfo.deviceType == DeviceType.Handheld) {

			gohere = Input.acceleration;      // acceleration vector from mobile device's built-in gyro

			if (gohere.sqrMagnitude > 1)
				gohere.Normalize ();
			
			horiz = gohere.x;
			vert = gohere.y;
		
		} else {
			horiz = Input.GetAxis (xaxe);     // use the default Unity input device, possibly a mouse or joystick
			vert = Input.GetAxis (yaxe);
		}

		moveMe (horiz, vert);

	}

	void moveMe (float horiz, float vert)     // adds the movement vector to the current ball vector
	{
		Vector3 moveme = new Vector3 (horiz, 0.0f, vert);
		
		GetComponent<Rigidbody> ().AddForce (moveme * speed * Time.deltaTime);
	}

	void OnTriggerEnter (Collider other)      // triggered whenever the ball collides with another object
	{
		if (other.name.Contains ("Target")) {           // was the object one of the Target cubes?
			GetComponent<AudioSource> ().Play ();   // play happy sound (Homer Simpson saying "Woo-hoo!")
			Destroy (other.gameObject);             // obliterate that cube
			count++;
			SetCountTxt ();
		}                                 // "else" could be one of the walls, if we wanted the walls to actively influence the game
	}

	void SetCountTxt ()
	{
		countTxt.text = (maxcount - count) + " mini-cubes to go!";
		if (count >= maxcount) {                  // all cubes are gone
			countTxt.enabled = false;         // disable counting text visibility
			winTxt.enabled = true;            // show winning text (its value is set in the Unity scene editor, in this case "Nice going!")
		}
	}
}