Wednesday, September 23, 2015

We've Come a Long Way Baby - WriteableBitmapEx now on GitHub

The WriteableBitmapEx project is now 6 years old and times have changed since all began. It all started with Silverlight and more XAML flavors were added over time until all XAML flavors were supported including Windows Phone, WPF, WinRT Windows Store XAML, (Windows 10) UWP and still Silverlight.

The version control system and open source services landscape has changed as well and the CodePlex infrastructure has not been too reliable recently. That's why I finally took the time to move the WriteableBitmapEx source code from CodePlex' SVN bridge to Git and make it available on GitHub
Additionally was the open source license changed from Ms-PL to MIT which should simplify the usage as well. See the Codeplex Issue Tracker for a list of features that will be added in the future. Please use the GitHub Issues functionality to add new issues which are not already reported. Of course the latest binaries will continue to be distributed via NuGet.

Friday, May 22, 2015

My Top 5 Wishes for Microsoft Band and Health

You might have seen my previous blog post ZzzZzz - Microsoft Band Sleep Tracking in the Testbed. In that article I compared the Microsoft Band and Health data to a proven scientific method for sleep monitoring. While writing it, a couple of ideas and suggestions came to my mind for the Band in general and some particular for the Sleep tracking feature.

Here's my Top 5:
  1. Pulse Oximeter:
    Oxygen saturation is a key metric for medical analysis. For example the result of sleep apnea can be seen as a drop in oxygen saturation and many other dysfunctions surface through that metric. It's also interesting for fitness and sports in general and some fitness trackers actually provide this data already.
    The Microsoft Band measures the heart rate optically using a photo detector and this is the same principle used for Pulse Oximetry, so the sensor is already there. I think it's just a matter of adding the right algorithm and surfacing the data in Microsoft Health.

  2. Sleep tracking Precise mode:
    I would really love if the Band's sleep tracking had a Precise mode which would take samples more often during a night and was allowed to run down the battery in 8-10h, so I can decide when the precision-battery-tradeoff is applied.

  3. Re-occurring alarms:
    The vibration alarm is awesome but it needs to have a feature so it can be made re-occurring. The Band could just sync with my Windows Phone alarms but should also work when the Band is not connected to the device via Bluetooth.

  4. Smart alarm:
    Although the sleep stages are not 100% reliable it would still be good enough to wake the user up during a certain timeframe when the sleep is not Restful anymore. Lots of fitness trackers provide that feature already which actually have less sensor data.

  5. Notifications:
    Currently the Notification Center and Mail tiles stay at the count until the tiles are opened. This is rather annoying as it is another notification hub which needs to be cleared in addition to the phone. I'm sure it's a technical challenge with the WP APIs but for vNext it would be great if the Mail and Notification Center tile count would also be synced over Bluetooth when those are cleared on the phone.

ZzzZzz - Microsoft Band Sleep Tracking in the Testbed

I'm a perfectionist and hungry for knowledge, so I went to a sleep clinic just to check how well the Microsoft Band sleep tracking is working compared to proven scientific methods.
Unfortunately it's not like that and I actually had to go to the sleep lab since I have some issues there. Anyway, since I had to go to the sleep laboratory in the clinic and I just got my Microsoft Band I took the occasion and also used my Band's sleep tracking feature while being monitored using the lab's Polysomnography sleep study.
After my 2 nights at the clinic I asked the lab staff and physician if I can get some personal printouts of my results. They were all nice people and the printouts provide some good data for comparison with the Microsoft Band and Health data.
A funny anecdote: The lab's senior physician is actually a big Windows Phone enthusiast and even tried Windows 10 for Mobile on his device so we had some good chats, also about HoloLens and how it can help them...

The science of sleeping
Sleep studying is a huge scientific area with a lot of active research and many parts of sleep are still unknown territory. I'm not a sleep research expert but learned a few things recently.
There are a couple of different methods for sleep monitoring / tracking. The gold standard for sleep labs is the Polysomnography (PSG) which uses a bunch of sensor data like EEG, ECG, oxygen saturation, eye movement, breathing sensors, muscle movement, etc. For this 2-3 dozen cables are connected to your body for a PSG sleep study.
Another, more basic method is called Actigraphy which just tracks the movement and maybe a few other parameters so it's only used to get information when a person went to bed and when they got up but it can not provide solid data about sleep stages as those depend mostly on your brain activity. The sleep features of fitness trackers like the Jawbone Up, Fitbit, etc. are basically actigraphs. The folks of Exists wrote a very good blog post about those devices and also link to some research studies about actigraphy and how well it works in comparison to PSG. The Microsoft Band also uses actigraphy plus some additional sensor data like samples of heart frequency. Maybe it's even using the skin temperature sensor as well.

Science basically defines five stages of sleeping: The phase when you are awake (W), the rapid eye movement (REM) phase when you are dreaming and 3 Non-REM phases where NREM1 (N1) is the transition to sleep, NREM2 (N2) is light sleep when the heart rate starts to decrease and NREM3 (N3) defines deep sleep when you are fully relaxed and difficult to awaken.

Enough of the theory...

Analysis of Night 1
The first night at the lab was pretty bad. All those cables and a clinic bed were very distracting so my actual sleep was very short and the phases not at all optimal. Keep this in mind when looking at the graphs. My home sleep data is usually way better.
In the table below I listed the parameters the Band provides and compared those to the Lab data. I omitted the total Duration (TIB), Time to fall asleep and Sleep efficiency since the lab and Band measuring started/ended at different times and those parameters depend on that. The Resting heart rate is also excluded as no clear comparable data is available in the PSG printouts, but looking at the graphs the Band Resting HR is somewhat correct.

Parameter Band Lab PSG Band error [%]
Actual Sleep (TST) [min] 257 319.5 -20%
Light Sleep (N1 + N2 [min] 184 271.6 -32%
Restful Sleep (N3) [min] 73 30.3 141%
Woke up 10 9 11%

As expected does the Band actigraph not provide as accurate data as a PSG and the sleep stages like Light sleep and Restful sleep don't match the actual scientific sleep stages very well.

In the below illustrations I overlaid the Microsoft Band's Health sleep charts (colored bars) with the PSG sleep stage graph (black lines). The black scan of the PSG printout was scaled to match the correct time and size of the colored Band data.

Night 1: Colored Band stages data overlaid with black PSG stages

This overlay is pretty interesting and tells quite a bit. Note the Wake phases match mostly but since the Band sampling only happens every few minutes it does not record higher frequency changes and therefore misses short naps or awake periods.
The REM phase is considered Light sleep by the Band which makes sense. Band's Restful sleep is mostly N3 but also a good portion of N2. I assume the Band only uses the heart frequency to define the Restful stages and since the heart rate starts to decrease in N2 the band labels the Restful stage sometimes in N2 until the body goes into N3. The next two overlays underline this theory.

Night 1: Colored Band stages data overlaid with black PSG heart rate
In the above illustration the PSG heart rate was overlaid on the Band's sleep stages and the heart rate chart. As can be seen the Band's Restful blocks correlate with a longer, lower heart rate period.

The third illustration below has the PSG heart rate on top of the Band's heart rate chart.

Night 1: Band's heart rate data (thick purple line) overlaid with black PSG heart rate

Again, the Band's low sampling frequency for sleep mode does not record higher frequency changes like peeks in heart rate which can be caused by sleep apnea when the body tries to compensate low oxygen saturation. Improving those measurements is one of my top wishes / suggestions for the Microsoft Band / Health service.

Analysis of Night 2
The second night was a bit better and I felt more rested after it. I guess mostly because I was super tired from night 1 and therefore slept better even with all those PSG sensors attached.

Parameter Band Lab PSG Band error [%]
Actual Sleep (TST) [min] 310 298.5 4%
Light Sleep (N1 + N2 [min] 183 199.3 -8%
Restful Sleep (N3) [min] 126 70.1 80%
Woke up 9 8 13%

If the Night 1 and Night 2 data is compared it can be seen that my subjective impression was not wrong. I got more than twice deep sleep (N3) during Night 2.
The Band vs. Lab PSG also seems to have less errors which leads to the conclusion that higher sleep quality produces better Band results. I'm not sure how the Microsoft Health analysis of the Band tracking data actually works but I wouldn't be surprised if it uses Azure's machine learning service and the used training datasets contain mostly healthy and normal sleep patterns.

Night 2: Colored Band stages data overlaid with black PSG stages

Again, the Band labels longer periods of N2 and always N3 as Restful sleep.

Night 2: Colored Band stages data overlaid with black PSG heart rate

Night 2: Band's heart rate data (thick purple line) overlaid with black PSG heart rate

The Microsoft Band sleep tracking feature is a nice gimmick but it's not really surprising it doesn't look great when compared to proven scientific sleep monitoring. The Band is not bad for the total sleep duration data, the actual sleep time and the wake up phases. The different stages like Light and Restful sleep are not terribly wrong either but should not be trusted too much. The analysis of sleep stages requires actual brainwave data which is not provided by the Band sensors. It seems the Band's Restful stage is mostly based on the heart rate and movement.
Of course the Band should not be used to diagnose sleep disorders like sleep apnea due to missing sensors but also the low frequency sampling in sleep mode. Hopefully this can be improved. Still if you feel tired more than usual and the Band / Health sleep patterns are totally off, go see a doctor soon!
In the end the Band is a fitness tracker and not a scientific measurement device. The Band and Microsoft Health's sleep tracking is still a nice feature to get an overall idea about your sleep patterns if you keep the limitations in mind.

Happy tracking!

Wednesday, May 20, 2015

100 to the next 100!

Oh wow, this is my 100th blog post here. Great things happened since I started this blog 6 years ago.
Most importantly were we blessed with our third and fourth daughters. I also got the Microsoft MVP award a few times, first as Silverlight, then Windows Phone Dev, now Windows Platform Dev and became a Nokia Developer Champion. I also joined IdentityMine almost 3 years ago. I judged the Imagine Cup 2011 and German App Campus, spoke at a few conferences like //build 2014, 2015 and wrote a couple articles for different magazines. I also published many releases of my apps and open source projects. And of course posted 100 blog posts with hopefully useful content.

Busy times but good times. I hope this goes on for a while.
Thanks for being part of it!

Tuesday, May 12, 2015

Bridges connect the world - iOS and Android on Windows 10. What?

Bridge to Astoria, OR. Nov 2014
Photo from our road trip to Portland
At the //build 2015 conference Microsoft announced their development stories for Windows 10. This also includes bridges to reuse existing iOS, Android and other investments for Windows 10.
The keynote showed how an existing iOS game was ported to Windows 10 without much effort. The shown one-stop porting solutions for Android and iOS raised the concern why anyone should bother to develop natively for Windows 10 anymore and created quite some confusion and uncertainty in the Windows developer community. A few developers at //build were really worried about those announcements as some of their business is mainly porting iOS and Android apps to Windows Phone.

I think at the moment there's just too less knowledge to make decisions based on the announcements and assumptions floating around. There were two sessions at //build about Project Astoria for Android and Project Islandwood for iOS. Keep in mind both projects are in a alpha/beta stage and aren't publicly released at the moment so there's no public hands-on experience and independent in-depth analysis available yet. It's also worth noting that Project Astoria allows one to build Windows apps for phones but not for all Universal Windows Platforms at the moment. More information is expected during Summer 2015 and you could also try to sign up for an early access program for Astoria and Islandwood. There's also a good post by my friend Alan from AdDuplex. Peter Bright published a good Ars Technica article as well.

In my opinion there's one particular really good use case:
Share existing business logic across platforms and build the UI natively for the platform.
This way a common cross-platform app core is reused and the UI is built natively on top of that layer to provide the best UX. Porting an app completely might work for games with a custom UI and some simple apps, but advanced apps usually leverage too much platform-specific features for a one-stop port to work at all and a great app adapts to the UX of the platform.
Sharing core business logic is actually a common use case I see with our larger clients who want to bring their existing apps to Windows or Windows Phone. In the projects I've been involved over the last few years, the shared portion was usually implemented in C/C++ as it is still one of the best cross-platform options since most platforms support C/C++ in some way. Right now it is still quite an effort to get the custom C/C++ libs from other platforms adapted for Windows Phone compilation/linking, even with elevated API access rights. The cross-platform C++ Clang support in Visual Studio 2015 allows us to just recompile the Android or iOS C/C++ code and could help us a lot for such scenarios.
After writing this blog post I came across this excellent post by the Visual C++ team which describes just that. And this post which provides more details about the Visual Studio 2015 cross-platform C++ features.

As I see it, Xamarin will be the best choice for green field cross-platform projects to share as much code as possible and still provide the best UX. The new Visual Studio 2015 tooling for Windows 10 could help to make the reuse of existing iOS and Android business logic easier for Windows 10 when there's not a desire for a green field.

On the last day of //build I had a nice breakfast and roundtable with an executive of the Windows Dev Platform and of course we talked about the //build announcements. There was actually the same vision about the easier reuse of business logic. I think we made it clear that quite some confusion was created in the dev community around Islandwood, Astoria but we were told Microsoft will release more information to make the vision for those porting tools clearer.

In the end I think there's no need to be worried. Porting tools have never been a one-stop solution so I'm confident Islandwood and Astoria won't be the holy grail either. As usual a keynote also contains a good portion of marketing and software development is never that easy, so always consume such announcements with a grain of salt and wait for final decisions until the expected is delivered for real-world scenarios.
If those tools help us to make the porting of existing iOS and Android apps easier and the Store provides the apps which users are looking for, it's good for all Windows developers in the end.
It's just a matter of adapting to leverage the new tooling and helping clients to create great apps.

Don't Worry, Be Happy!