Apr 05, 2018

I'm not ready for Jupyter Lab

It's the latest thing from the people that brought you Jupyter Notebooks, but it has a few quirks that I can't seem to get used to. Chief among them is that in the interest of security, they've blocked inline Javascript from executing. This manifests itself when you try to run old notebooks and get this, instead of the graph you were expecting:

JavaScript output is disabled in JupyterLab

I'm all for security, but breaking existing files this way is unacceptable. The boilerplate from the Jupyter people is,

"The official, recommended way for people to provide their own custom outputs, widgets, etc is to write a JupyterLab extension."

So what this means is that everyone who already had working implementations of plotting features in Jupyter Notebooks has to go write another extension for them to work in Lab. It also requires the user to install additional stuff as well for it all to work. While this isn't too difficult, it's a pain in the ass AND not all packages have extensions written already.

The new Lab interface is very clever, possibly too clever for me. They've tried to incorporate all sorts of useful features to make it more like an IDE, but this comes at the price of having to learn a more complex UI than the old Notebooks had. It's all well done, but I find the explicit functionality less enjoyable to use than the beautiful, stripped down elegance of the old Notebooks, which surprised and delighted you as you used them. Personally, I prefer using a real IDE, if I'm going to code something, but if you're looking for a browser-based IDE, Jupyter Lab is probably something you should look at.

Jupyter Lab Interface

Jupyter Lab Interface

I'm certain that at some point the old Notebooks will be deprecated entirely, so I'll keep poking at the new stuff, but until either the Jupyter people figure out an easy way to enable old functionality, or all the various packages I use have Lab extensions written, I'm not ready to use it full-time. Hopefully support for the Notebooks will continue for some time to come.

Mar 17, 2018

My Last Day at Tableau

Yesterday was my last day working at Tableau Software. After 6 1/2 years of doing pretty much the same job as when I started, I was feeling a bit stale and ready for a change. Fortunately, I didn't have to go far to find an area of interest. The quantitative analysis and visualization of data has fascinated me since the first time I read Tufte and saw Minard's map of Napoleon's March. It's the reason I went to Tableau in the first place, the idea that maps are just another way to visualize data. I'm looking forward to taking a few classes, doing some consulting work and solving some linear regression problems that Tif's given me to ponder .

I've read many emails from people who left before me expressing sadness at their departure. I leave feeling relief and optimism about finding new, engaging problems to solve in the future. I leave feeling fortunate to have met - and worked with - a handful of exceptional people whom I truly admire and respect (Maureen, here's looking at you, kid). I leave feeling proud of a few things I accomplished; bringing about some order and maintainability to the Geocoding database and inspiring and helping to build a mapping service that worked "pretty well" for 3+ years. And I leave feeling lucky to have participated in the refinement of a product that is now viewed as an industry standard. But none of these things is enough to make me want to stay one-more-day at Tableau.

It's been clear for a long time that if I wanted work worth doing again, I would need to start looking at opportunities outside the narrow confines of my current position. I'm now on a quest to find that again and I wish all of my former colleagues the very best. I look forward to seeing all of you from the user side of the house in the future.

Mar 3, 2018

Making a Map - Where to start?

A sentiment I've heard expressed a few times recently and one which I've felt myself is, "Where do I start?". With so many options in mapping technology available now, all slickly marketed and hyped to death, it can feel overwhelming for someone trying to get started. The answer, of course, is to start from the beginning.

What are you trying to show, and how do you want to communicate it to your audience? These 2 things should help you decide what tool or platform to use. Here are some questions to consider related to these.

  1. How much of the globe and what part of it are you interested in showing?
  2. What do you need to show?
  3. How are you going to style your map?
  4. How are you going to present your map?

How much of the globe and what part of it are you interested in showing?

This makes a difference with regards to the map projection that will work best for you. Don't know what a map projection is? No problem. Find a tennis ball and a flat piece of paper. The ball is the Earth, the paper is your computer screen. Now wrap the ball with the paper. Doesn't look like very good, does it? A map projection is the shape-shifting mathematical voodoo that allows your rectangular computer screen (and in ye olden days, a paper map) to display a section of (or the entire) ball in a way that doesn't necessitate you dropping acid to make sense of what you're seeing. The more of the ball that you want to show in your map, the more the projection matters.

Map Projections

How this junk works

The other factor I mentioned here was what part of the globe you're trying to show. This matters, again, because of the map projection. Basically, the closer your area of interest is to the poles and the bigger that area is, the fewer of the NewMappingHotness.com options will work for you. This is because most of them only support 2 projections that start to look like crap when you get North of Edmonton, CAN. Pretty much the hardest thing to show in NewMappingHotness is something like a map of polar ice recession. You can see why pretty easily, just fire up Google Maps and do a search for, "The North Pole". Good luck with that. It's not that Google Maps is bad, it's just that in order to give you a map that let's you zoom in to (almost) any other part of the ball, something has to give and it's the Poles. (BTW, Bing maps will at least try take you there, but it's still goofy looking.)

What do you need to show?

"A map", you say. Well yes... but what things do you need on the map in order for you to tell your story? Everything on your map should be there to help tell the story. Look at these maps of murders in Chicago and New York by the NY Times. Everything on them helps the reader to see where most murders take place, or where most segregation exists. There is only enough data on each map to tell these stories and nothing else. Roads, parks and water aren't labeled and there's only enough information there for a reader to make out where in the city they are looking. These are beautiful maps that do their job well. In order for your map to be effective, you need to identify the geographic features that you think are needed. And that means that anything that you don't explicity identify has to go. This will eliminate some of your options, as only a few of the NewMappingHotness.com platforms allow you to turn layers off.

How are you going to style your map?

This is entirely a function of what you're trying to show and what features you think need to be on the map to provide context. For example, if I were going to make a map that showed all the public bathrooms on Naxos (Greek island in the Aegean), I would need to style the map differently than the Murder Capital map I mentioned above. I would need to make the bathrooms very visible in contrast to the rest of the map, and I would need to have enough other features on the map that people could find the bathrooms, and I probably would need some labels as well. And what if there were different kinds of bathrooms and I wanted to color them differently, maybe "flush" versus "hole in the ground"? And what if I wanted to also call out that some were "one-holers" versus "party-poopers"? Maybe I'd use color to indicate type, and make the size of the dot proportional to the number of stalls, or something like that. This again is something that will help you decide what tool to use. Some make this sort of thing easier than others.

And finally, how are you going to present your map?

Is it going to be in a Web page, maybe on a blog that gets read both in mobile and other browsers? Or is it going to be a figure in a PDF document that gets scaled up and down in size when it's read? Or maybe it's actually going to be printed on paper? (That bathroom map isn't very handy on an electronic device while you're wandering around Naxos on foot.) Probably it will be a combination of those things. The point being that you might need different output formats and not all mapping platforms support them.

So to sum up this monologue, before you implode from trying to decide which of the myriad of cool new toys to try out, figure out what you actually want to make first. Once you have an idea, look for examples that are similar and see if you can determine how they were made. Trust me, you're not that original. Most of the really cool maps were made before computers and we're all just trying to catch up to what they made back then.

In my next post, I'll go through the process I described here and will make a map using a few different tools.

Feb 5, 2018

Running R in Linux Anaconda on Windows WSL

I prefer running R in Jupyter over using RStudio - no contest. I get intelligent formatting, direct access to help files and nice looking output which I can easily save to HTML and share. By far the easiest way to get Jupyter running is to install Anaconda. There are installers for WIndows, Mac and Linux and I've tried both the Windows and Linux binaries. Personally, I find that using the Windows binaries for Anaconda is as annoying as using Git for Windows. I just can't see the point of having all those specialized cmd shells for each program. I much prefer just getting the Linux installer from Continuum and installing it in the Ubuntu WSL environment that's available in Windows 10 now. Here are the steps to do that.

NOTE: This assumes that you have already enabled the Windows Subsystem for Linux (WSL).

I. Open your Ubuntu terminal by hitting the Windows key and "R" at the same time and type, bash.

II. Downloaded the latest Anaconda installer for Linux from Continuum:

    $ wget https://repo.continuum.io/archive/Anaconda3-5.0.1-Linux-x86_64.sh

III. Install the file by running it and deciding where you want to install it. I installed it in /usr/local, which means that you'll have to execute the installer as sudo and specify a different install directory than the default. (The default option works just fine though.)

    $ sudo bash ./Anaconda3-5.0.1-Linux-x86_64.sh

You'll be prompted to accept the license agreement and then to either install in the default location, or in a place of your choosing. Either way, at the end of the install you'll be asked if you want to add the path to anaconda in your local path. Unless you know otherwise, say "Yes".

IV. Optional: Add the anaconda3/bin path to securepath with visudo so that you can install conda pkgs later using sudo: (This is not needed if you installed to the default location.)

    $ sudo visudo
Add "/usr/local/anaconda3/bin" to securepath line.

V. Install some packages needed to make labels appear on R plots:

    $ sudo apt-get install libxrender-dev
    $ sudo apt-get install libxext-dev
    $ sudo apt-get install libcairo2-dev

VI. Install R essentials and an R kernel:

    $ sudo conda install r-essentials
    $ sudo conda install r-irkernel

VII. Run it!

    $ jupyter notebook --no-browser

[I 07:21:31.271 NotebookApp] JupyterLab alpha preview extension loaded from /usr/local/anaconda3/lib/python3.6/site-packages/jupyterlab
JupyterLab v0.27.0
Known labextensions:
[I 07:21:31.288 NotebookApp] Running the core application with no additional extensions or settings
[I 07:21:31.331 NotebookApp] Serving notebooks from local directory: /mnt/c/Code/data_science
[I 07:21:31.333 NotebookApp] 0 active kernels
[I 07:21:31.334 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/?token=11da29db9d864e64a9377a5a479c4b98a52c32bc286c959a

Jupyter running in Edge on Win 10

Jupyter running in Edge on Win 10

Feb 3, 2018

Excel's Online Helpfiles

I recently had a look at the official Microsoft help site for Excel. It's fantastic! If you use Excel and haven't seen it before, I highly recommend it. It gives you access to all of the tutorial and help files that are exposed via mouseover tips inside the product. The added bonus is that it also has a good search function and you can see the information in a separate window, so you don't have to waste your screen real estate inside Excel.

Excel helpfile

Excel helpfile

January 30, 2018

Notes, Notes Everywhere!

TL; DR - I tried Simplenote and I really like it. Depending on what you want to do, Evernote might have features that you'll like.

I'm a note-taker, can't help myself, if I'm honest about it. I have paper notebooks all over the place with various thoughts, reminders, ToDo lists etc. I've extended this habit digitally as well and keep a "Work Log" in which I record the things I've done during the day. This is especially useful to record any tech snippets that I discovered while doing something new. This has been a lifesaver when I've had to redo something esoteric and which I'd already forgotten. (Which seems to happen more and more these days.) Recently, I've been taking classes, both Online and in person at UW, and have wanted to keep track of topics unique to each class in separate notes. These notes are then used to help complete assignments and also to synthesize the information that's been presented in each class. And finally, many of the ad hoc notes that I've made end up getting converted into HTML files and used as HowTos and user docs. These various needs have exposed the inefficiencies and shortcomings of my existing/old process.

Up until now, I stored notes as plain text files in Dropbox, or on GitHub. Dropbox kept them in sync and made them easy to access from different machines, but I had to have Dropbox running locally in order to edit them and sync the changes. It also has only lukewarm support for Markdown syntax. In particular, code blocks don't seem to be rendered properly. GitHub supports Markdown extremely well, but who wants to store their whole life on GitHub? I like it for documentation that accompanies a project that's stored there, but for personal notes, especially ones that I want to edit on my phone, GitHub isn't a viable solution. So what else is out there?

Turns out that there are a ton of options available. Two alternatives that I've tried out this week are Evernote and Simplenote. Both have in-browser and installed apps and allow you to organize notes, but beyond that, they are radically different.

Evernote

Evernote tries hard to be an all-in-one document editor. It has options to control font style and size, whether text is centered, left or right-justified etc. You can export to PDF, embed web pages or images in the doc - heck you can even embed an audio recording. Here's an example of an Evernote page saved as a Web page. An additional feature it has (which Simplenote lacks) is the option to create notebooks to organize notes. For me, it borders one being more than I need and it suffers from 2 deal-breakers.

  1. It doesn't support Markdown, eschewing it instead for built-in formatting controls like you'd see in Word.
  2. It only supports 2 devices before you have to pay for it.
Screenshot of Evernote interface

Evernote screenshot

I can live with the pay wall concept - heck I even support it for something that I like and use, but the lack of Markdown is a problem for me.

Simplenote

Simplenote is far less flashy than it's counterpart. It pretty much gives you what it implies - a simple interface for making text-based notes and storing them. But where it really shines in comparison to Evernote is in it's built-in Markdown support. Here is a note written up in Markdown and then published from Simplenote to the web. It lacks the Notebook feature Evernote has, but by using tags, you can easily accomplish the same thing. I've tried the Web, Android and Windows apps and they all seem to work really well. An added bonus that I've captured in the screenshot above is the "Dark" mode setting which mimics the black background found in most of today's code editors.

Screenshot of Simplenote interface

SimpleNote screenshot

So for now, I'm sticking with Simplenote.

January 28, 2018

Time for a new blog

2018 feels like a year for new beginnings and a good time to clean out old junk. In that spirit, I've decided to give my old blog a kick in the pants. A few of the goals I have in mind for this incarnation are:

  • Simpler page creation process to enable faster publishing
  • Cleaner, more flexible layout that responds well to mobile devices
  • Leveraging the github.io domain for Web page hosting
  • Organized storage of one-liners and snippets for personal use
  • Sharing of useful tech-related insights
  • Updates on hobby projects for friends and family
  • Rants, let's not forget about the rants.

I've decided to use Pure.CSS as the basis for the site. I don't know much about the project, but the pages look nice and the CSS seems to respond to my crude manipulations. Editing and creation of base entries will be done in Atom.io using a couple Markdown packages to generate GitHub flavored Markdown, which will then get jammed into templates. (See "crude CSS manipulation" above.) We'll see how it goes, but hopefully it'll all "just work". And if not, well it's all just a learning experience anyhow.

Short, announcement-style, entries will just get tacked above this one, organized by date. Longer and more expository pieces will get filed under the "Sections" button to the left. It's disabled at the moment, but hopefully I'll have some stuff in there by the end of this week.

Feel free to leave me feedback, or point me to intersting things via Twitter.