on{X} update

The developer community’s response to on{X} has been really fun to watch.  I’ve made a short post for the site about some of the highlights of the past week.  In late-breaking news, someone has apparently just whipped up a fledgling independent on{X} marketplace.  It’s immensely satisfying to make something which in turn is used to make.


Posted in mobile | Tagged , , , , | 5 Comments

on{X}

A couple of weeks ago I came back from a great trip to visit my very talented and hip team at the Microsoft Israel Development Center in Tel Aviv.  Excitement was running high as we geared up for the launch of on{X}, which I can now talk about, as it went live today.  Here’s the “trailer”:

 

Eran, the team lead, already wrote a nice and detailed blog post for the website talking about on{X}’s capabilities, so probably I should just stick with the link.  I can’t resist reiterating some of his points, though, and giving a bit of my own color.

Sometime in the middle of the last decade, it became clear to everyone that mobile phones aren’t appliances or pieces of single-function consumer electronics, but rather are little personal computers— small enough to keep in your pocket, and with just enough battery to stay on the whole day.  I think that because of a kind of metaphor hysteresis, the implications of this still haven’t fully sunk in; we still call this device a “phone”, which is the same name we give to the fixed-function radio handsets some of us still have docked in a cradle on the kitchen counter… which are in turn nothing but “cordless” versions of their midcentury forebears, which in turn consisted of nothing but a speaker, a microphone, some wire, and some kind of dialing apparatus encased in a Bakelite shell.  Even that silly word we use in America, “wireless”, sounds like more or less a synonym for “cordless”.

So we now agree that it’s not really a phone, but rather a telephony-capable pocket computer.  To call it a “phone” is an unnatural act of metonymy akin to calling a toolchest a screwdriver.  Think of how funny it sounds to say “my phone comes with a phone app preinstalled”.

One oddity about this pocket computer idea is that unlike the bigger computers those of us who like to hack have grown up enjoying, smartphones still inherit much of the fixed-function thinking that pervades consumer electronics.  Deep in the phone’s OS is a kind of dispatching or command-and-control system, a switchboard, that takes all of those wonderful sensors and capabilities and organizes them into rigidly predefined appliance-like behaviors: when the always-on radio detects an incoming phone call, launch the phone app.  When there’s an incoming SMS, launch the SMS app.  You don’t need to know that there’s an app for that, because the phone itself does.

What if you could open up that switchboard on your own device and rewire it to do whatever you want?  With all of those sensors and capabilities, and all the power of the Internet, one should be able to do a lot more than perform fixed phone-like functions and run apps.

This is especially interesting when it involves the automation of actions based on events on the phone or in the world, which one could call “push” or “reactive” behaviors.  (This is to distinguish them from “pull” behaviors, which are characterized by beginning with an explicit user action with intent— for these cases, the app model works well.)  We think there’s great untapped potential in push and reactive programming.  The sample scenarios and scripts we’ve put on the site begin to explore the possibilities, but we imagine that the developer community will come up with a much, much larger set.  That’s why we didn’t restrict on{X} to prescripted rules, but rather made it possible for anyone with JavaScript skills to hack a new behavior.  This thinking (and of course our choice of scripting language) is very much inspired by node.js.

The most exciting thing about this project, for me, will be seeing what people do with this wide-open field.  Makers, have at it!

In the meantime, we’ll be busily adding capabilities and sharing ideas.

3am update: apparently we’ve just made AppBrain’s top 10 hottest Android apps :)


Posted in mobile | Tagged , , , , | 4 Comments

translator

I was pleased to see David Pogue’s positive review of the new Windows Phone, Nokia’s Lumia 900, a couple of weeks ago in the New York Times.  Windows Phone has made great progress these past couple of years, and has advanced a beautiful and fresh design language, Metro, which we see being adopted all around Microsoft.  (I’ve been a big advocate for Metro language and principles in my own part of the company, Online Services.)  Pogue’s only real complaint is that apps for Windows Phone  are still thinner on the ground than on iPhone and Android— though as he points out, what really matters is whether the important, great and useful apps are there, not whether the total number is 50,000 or 500,000.  Many apps doesn’t necessarily imply many quality apps, and most of us have gotten over last decade’s “app mania” that inspired one to fill screen after screen with run-once wonders.

What really made me smile was Pogue’s characterization of what those important apps are, in his view.  After reeling off  a few of the usual suspects— Yelp, Twitter, Pandora, Facebook, etc.— he added:

Plenty of my less famous favorites are also unavailable: Line2, Hipmunk, Nest, Word Lens, iStopMotion, Glee, Ocarina, Songify This.

Even Microsoft’s own amazing iPhone app, Photosynth, isn’t available for the Lumia 900.

I’ve also been asked (a number of times) about Photosynth for Windows Phone… hang in there.  A nice piece of news we’ve just announced, however, is a new app for Windows Phone that I hope will join Pogue’s pantheon, and that is considerably more advanced than its counterparts on other devices: Translator.  Technically this isn’t a new app, but an update, though the update is far more functional than its predecessor.

Translator has offline language support, meaning that if you install the right language pack you can use it abroad without a data connection (essential for now, I wish international data were a problem of the past).  It also has a nice speech translation mode, but what’s perhaps most interesting is the visual mode.  Visual translation is really helpful when you’re encountering menus, signs, forms, etc., and is especially important when you need to deal with character sets that you not only can’t pronounce, but can’t even write or type (that would be Chinese).

Word Lens, mentioned by Pogue, was one of our inspirations in developing the new Translator.  What’s impressive about Word Lens is its ability to process frames from the camera at near-video speed, reading text, generating word-by-word translations, and overlaying those onto the video feed in place of the original text.  This is quite a feat, near the edge of achievability on current mobile phone hardware.  In my view it’s also one of the first convincing applications of augmented reality on a phone.  However, the approach suffers from some inherent drawbacks.  First, the translation is word-by-word, which often results in nonsensical translated texts.  Second, there isn’t quite enough compute time to do the job properly in just one frame, yielding a somewhat sluggish feel; on the other hand the independent processing of each frame is wasteful and often makes words flicker in and out of their correct translations, just a bit too fast to follow.  For me, these things make Word Lens a good idea, and better than nothing in a pinch, but imperfect.

The visual translation in Translator takes a different approach.  It exploits the fact that the text one is aiming at is printed on a surface and is generally constant.  What needs to be done frame-by-frame, then, is to lock onto that surface and track it.  This is done using Photosynth-like computer vision techniques, but in realtime, a bit like the video tracking in our TED 2010 demo.  Selected, stabilized frames from that video can then be rectified and the optical character recognition (OCR) can be done on them asynchronously— that is, on a timescale not coupled to the video framerate.  We can do a better job of OCR and translation, using a language model that understands grammar and multi-word phrases.  Then, the translated text can be rendered onto the video feed in a way that still tracks the original in 3D.  This solves a number of problems at once: improving the translation quality, avoiding flicker, improving the frame rate, and avoiding superfluous repeated OCR.  It’s a small step toward building a persistent and meaningful model of the world seen in the video feed and tracking against it, instead of doing a weaker form of frame-by-frame augmented reality.  The team has done a really beautiful job of implementing this approach, and the benefits are palpable in the experience.

Use this app on your next visit to China!  I’d love to read comments and suggestions from anyone trying Translator out in the field.


Posted in mobile | Tagged , , , , | Leave a comment

bear pond espresso

Much has been written about the food scene in Tokyo.  In the end, maybe it’s just a very, very big city.  The “largest metropolitan area in the world”, says Wikipedia, and among the densest (at more than 37,000 per square mile).  I’m sure that if we plotted the number of Michelin stars per city against either of these variables we’d find a more-than-linear relationship, for the same reasons one finds such relationships with the number of patents or any other measurable correlate of creativity.

Well, on a recent trip to Tokyo I experienced this correlation firsthand.. or maybe I just ran into a very special talent.  Not that either of these possibilities excludes the other.

Morgan-san introduced me to this very cute neighborhood, Shimo-Kitazawa, just a couple of tube stops from our hotel in Shinjuku, and home to the extraordinary Bear Pond Espresso, with owner and apparently sole barista Katsu Tanaka presiding over his tricked out La Marzocco.

Some pictures first of the neighborhood:

  

The scale and proportions of neighborhood architecture in Tokyo are so distinct.  I feel as if, were a streetscape reduced to nothing but featureless white boxes, I could still pick out a street in Tokyo from one in any other city in the world.  When it works, there’s something pleasing and intimate in this configuration of space, a sense of implicit trust, of social contract and efficiency, of highly intentional design.  In Shimo-Kitazawa the Japanese love of texture and material is also evident, and thankfully there isn’t so much of that ubiquitous rectangular ceramic tile on the building façades which I find so oppressive.  Instead, lovingly rusted iron plates, salvaged twists of wood and artfully rough calligraphy decorate the boutiques, which rub shoulders with what look to me like traditional Edo houses among the back streets.

And now, Bear Pond.  There was a “no camera” sign inside, so I was only able to snap a picture of the white sliding doors from the outside.  The space is small and cement-floored, combining the shabby genius sensibility of a Bay Area garage with the curated aesthetic Miyazaki and Kondō capture so wonderfully in Whisper of the Heart.  There’s enough carefully deployed clutter in the miniature space to make it clear that this is a neighborhood coffeeshop and not a boutique, yet even the furred, ingrown corners of the “order here” sign seem to signify, like the fulsome red drips of the graffiti on the coffee machine (courtesy Curtis Kulig).

     

(Note that several of these images are from the little Bear Pond book, which I bought a copy of although it’s undecodable by my non-Japanese-reading self.)

OK, let’s talk about the coffee.  It was amazing.  Morgan and I had an espresso and a macchiato each.  The espresso redefines ristretto.  It was a messy dark syrup, just a few cc’s, and dense, dense, dense.  There were plenty of trapped aromatics, but the thick extraction didn’t create anything like the usual tiger-stripe crema— at the bottom of the cup, nothing but an inky, intense depth of flavor and color, sweet, funky and complex, cocoa-ish, with no hint of the “southern” burnt flavor and even less of the acid lemoniness of an improper extraction.  This really was something new, and I’m afraid I don’t really have the words for it.  It’s just a different beast.

The macchiato was good— I would have been happy with it under any other circumstance— but I think it’s a waste to blend such exquisite nectar with milk, even small amounts.  The caramel-like effect Vivace achieves in the macchiato with their Vita blend was not in evidence here; this espresso felt both too dense and too ephemeral somehow to fold with the milk flavors, they seemed not impedance matched.  Perhaps an issue of pH?  The Japanese milk is different, also, with less body it seemed to me, and this may have contributed to my impression.

The workmanship of the little rosetta was of course perfect, but alas, “no camera please”.

In any case, if you find yourself nearby, treat yourself well— wander the neighborhood, drop in, taste what I suspect might be the most beautifully crafted espresso anywhere.


Posted in food, travel | Tagged , , , , , , | Leave a comment

fish stew

As I’ve been compelled to email this very good bouillabaisse-like recipe from Ondi to myself a number of times over the past couple of years, it seems more convenient and sociable to put to put it on the blog.  Here goes.

  • An onion.

Fry in olive oil.  Add:

  • Cumin seed
  • Coriander seed
  • Several sprigs of thyme
  • 2 bay leaves
  • Fennel seed
  • Several cloves garlic (pressed or finely chopped)
  • Saffron
  • Sea salt to taste
  • Pernod.

When onions are golden add:

  • 1 can of San Marzano DOP tomatoes (can be chopped beforehand, and can substitute fresh Roma tomatoes if sufficiently good ones are available)

Cook down for a good 20 minutes.  Then add:

  • Chicken broth or fish stock till the base is as thick or thin as you like.
  • When that’s cooked about 20 minutes, add assorted seafood.  Typically:
  • Monkfish
  • Salmon, in chunks cut from a steak is easiest
  • Spot prawns
  • Clams
  • Mussels

I use the prawn shells and heads to make the stock.

For extra joy, I also add sea scallops at the end, which I’ve seared in butter and flamed with Pernod.

This should be served with a crusty bread.


Posted in food | Tagged , | 2 Comments

escape from camp 14

Our good friend Blaine is gearing up for the publication of his new book, Escape From Camp 14.

It’s about a young man who has managed to escape from the most brutal of North Korea’s giant concentration camps, eventually making it to the US.  The story is absolutely harrowing.  North Korea’s total insularity seems to have kept most of the world from really understanding what’s going on inside— senseless horrors on a truly grand scale.  It’s a bit like imagining a Nazi Germany turned entirely inward rather than bent on dominating other countries.  In this scenario, would the rest of the world have stood by while the atrocities were committed?  The answer appears to be ‘yes’.

Over the past year, through lots of long family dinners and hurried hellos at the neighborhood coffeeshop, I’ve had the good fortune and honor to see Blaine develop Escape From Camp 14 from an idea, through several drafts, and of course through the ups and downs that go with this kind of project.  Blaine is a pro at this, and in addition to his very prolific career as a reporter, he has written some great books— A River Lost: The Life and Death of the Columbia (hurry!  Amazon says only 11 left in stock!), and Africa: Dispatches from a Fragile Continent, which Adrienne and I each read in about a day on an Africa jag that I remember also included Beryl Markham’s autobiography and What Is The What.  I have a feeling, though, that the timeliness of Escape From Camp 14, as well as the extraordinary source material, might make it his big bestseller.  I certainly hope so— this is a story that needs to be read by many.  It comes out next month, and you can preorder it now.

Blaine enlisted me to do the voiceover on his promo video.  He assures me it turned out OK, though I can’t actually listen to it.  There’s something horrifying about hearing your own voice recorded.


Posted in books | Tagged , | 1 Comment

invention

Before the five Equalizing pieces, I made an invention in d minor to begin experimenting with counterpoint:

Invention in d minor

This was the first thing I recorded on the little personal recording device.  It’s a bit of a mess, but posting it anyway, as it has some ideas that might be worth returning to when I’ve improved.  It’s recorded in two passes, since I can’t yet articulate all four voices in realtime (and this leads to some timing problems).  I used the harpsichord setting, both because it seemed appropriate and because in the beginning I felt unable to control the piano dynamics well enough.


Posted in music | Tagged , , , | Leave a comment

equalizing

So I’ll now take a deep breath and post the first edible fruit of my labor.  The performance is of course imperfect, since I’m not an actual pianist, and the piece should also be considered a draft, as there are places where it still needs editing.  But I do hope it manages to cross the threshold into music.

  1. Equalizing I (Tone pattern)
  2. Second phases
  3. Equalizing II (Clockwork)
  4. Night train
  5. Equalizing III (Indonesia)

OK, some liner notes.

Equalizing is in five movements, where the first, third and fifth are variations I, II and III on a central idea, while the second and fourth are interludes.

The ascending octave tone pattern in the variations is something from childhood.  It was an analog equalizer, one of our pieces of stereo equipment, which when put in tone generator mode would light up the sliders in sequence while playing the progression of octaves, from a bass you felt in your stomach to an inaudible dog-whistle.  Sometimes when we’re young we become obsessed with apparently trivial sensations— I remember watching our kids at the beach very slowly dribbling sand through their fingers for hours.  I’m not sure exactly what’s going on during these long unprompted meditations which would try the patience of a Zen monk, maybe it’s significant, maybe not.  But I certainly remember many of those times.  I remember the sense of flow, the engagement of all of the senses at once even when the input was purely visual, or purely auditory.  And I remember the limbic power of those intensities, the disembodied yet primary-colored emotion, hypnotic and unregulated.  Even the echo of that is a treasure.

Yes, I know I made overmuch use of the sustain pedal.

Connoisseurs of the WPRB school will notice the influences of John Fahey and Guy Klucevsek, as well as Steve Reich and Arvo Pärt.  Like the last two, I’ve tried to build around a stripped-down harmonic armature (though I haven’t been as rigorous about sticking to it as those sages).  This feels consistent with the octave tone pattern at the core of Equalizing, which on the one hand is trivial in that it lacks any harmonic structure at all.  On the other hand, containing nothing but powers of two, it’s neither major nor minor, but hovers behind and above this passing weather in a kind of mathy space beyond sad or happy.  Is it just me, or is this naked precursor to tonality already thrumming with feeling?


Posted in music | Tagged , , | Leave a comment

piano

A couple of months ago I began playing with our new digital* piano.  What a lovely thing a keyboard instrument is!  You just push a button, and out comes a perfect note— hard if you hit it hard, soft if you hit it softly, and never out of tune.  The buttons feel great and they’re laid out in a neat row.  The tonal range yawns across many octaves.  You’re limited only by the physics of your hands.  It feels like cheating compared to every other instrument I’ve ever played.  No wonder music began really taking off when this technology was introduced.

(*Why digital?  Because it has headphones.)

Not that I can be said to actually play the piano.  Over the past couple of months I’ve “learned” it in much the same haphazard way I learned to type thirty years ago— without any discipline or technique whatsoever, driven purely by the urgent desire to transduce.  (Now that I think of it, most of what I know is characterized by this rather sloppy approach.)  I’m sure I could achieve better results by studying it properly with a teacher, and I’m still painfully slow at reading from the staff.  But: I began making music anyway, with the piano, a handheld recorder, and some notation software.  There have been some late nights.


Posted in music | Tagged , | Leave a comment

treasure hunt

Over the past year I’ve set the kids a number of treasure hunts, which are both fun to make and always in demand.  Like anything one tries a few times, it’s gotten quicker.  This morning’s was assembled in record time and under duress.  It turned out decently given constraints and incorporated a shopping trip, though I’m regretting not having made Eliot’s half of it harder.  Next time..

The structure usually works like this.  Clues are written on slips of paper, which are hidden in various places, each pointing to the next.  There’s some sort of prize at the end.  These were the clues:

My notes:

The library is useful as a place to hide clues— it’s a source of fun things to read, and shelf column, row and book number form a coordinate system, which is good for math problems.

The Dylan Thomas poem was Where once the waters of your face, with the following clues in a jar of salt and in sage:

Thanks to Ilse for taking over from A5/E6.  She MMS’d me their favorite Schiele paintings from 1909-10,

.. after which I texted back the final clue leading to their prizes:


Posted in children | Tagged , , , | Leave a comment