Category Archives: mathematica

Mathematica 10

Things I particularly like about Mathematica 10.

1. Multiple undo

It undoes both the typing and the results of one’s typing. Yay.

2. Curated APIs

Mathematica can now natively communicate with a number of online services, tweeting or analyzing one’s tweets, downloading images from instagram, making association graphs of one’s facebook friends, and so forth.

I have in the back of my mind to set up an IP camera that Mathematica monitors, and that tweets when it detects specified activities. It would take only about three lines of code.

In practice, so far, I’ve been using it mostly with RunKeeper.

runkeeper = ServiceConnect["RunKeeper", "New"];
ServiceExecute[runkeeper, "UserData"];
id = First[runkeeper["FitnessActivities"]]["ActivityID"];

runkeeper["AnimatedPathMap", {"ActivityID" -> id,
AnimationRunning -> False}]


The major weaknesses here are (a) not every service that Wolfram connects to publishes the information that you want via their API. So, for example, while there is the great FriendNetwork from Facebook and FollowsNetwork from Twitter, there is nothing comparable from LinkedIn. Further, sometimes the service does make information available but Mathematica gives you no way to query it. For example, the RunKeeper connection would be about a million times more useful to me if I could pull HRM data as well as the geopath. RunKeeper can provide it, but Mathematica 10 doesn’t know how to ask. Perhaps if enough of us complain about its abence, Wolfram will respond.

3. Associations

One could create structured lists before, but this is vastly more flexible. Instead of

{name, age, street address}

where the form is rigid and can be confusing especially for long records with mostly blank entries, and where the form must be carefully documented for there will be any hope of reusing the data or the code that interpreted it, one can now do

In[59]:= item1 = <|name -> "bill smith", age -> 37, streetAddress -> "121 Main Street"|>;

In[60]:= Lookup[item1, name]

Out[60]= "bill smith"

In[66]:= addressbook = {<|name -> "bill smith", age -> 37,
streetAddress -> "121 Main Street"|>, <|name -> "susan smith",
streetAddress -> "121 Main Street"|>};

In[67]:= Map[Lookup[#, name] &, addressbook]

Out[67]= {"bill smith", "susan smith"}

One can also combine elements from multiple associations into a single record, something that would have been more difficult with the old method.

4. Wolfram Cloud deployment

This is not as good for Manipulate[] as the Mathematica plugin, but far more accessible.

In[8]:= CloudDeploy[
runkeeper["AnimatedPathMap", {"ActivityID" -> id, AnimationRunning -> True}],
Permissions -> "Public"]

Out[8]= CloudObject["\

Other thoughts

Templating and, in particular, automated report generation, would have been useful in my last job. I’m not sure about this one but it feels as though it may come in handy.

Wolfram has improved support for the standardized time series datatype that they introduced in, I think, version 9. I’ve been using a homegrown method since long ago and the Wolfram version is lacking a lot of functionality still (the ability to do math between time series, dividing one by another, for example. The ability to normalize a time series, etc.). Together with the ability to manipulate metainformation (introduced in version 9), this is now a clean way of handling financial information. I expect to start migrating my code over to use the new format.

IRR and NPV in Mathematica, the modern way

Back in 2010 I posted an explanation of how to compute net present values and internal rates of return in Mathematica. It worked but it was a little verbose. Considering that Mathematica can do things like optical character recognition with a single command, it seemed a shame that it couldn’t do these basic financial functions.

Wolfram has fixed this, and computing IRR and NPV in Mathematica is now much more civilized.

Using the same sample cash-flow as last time (but reformatting the date to list-format, as Mathematica’s Cashflow[] function demands for some reason),

testFlow = {{{2001, 1, 1}, -1}, {{2002, 1, 1}, .3}, {{2005, 1, 1}, .4}, {{2005, 1, 10}, .5}};

TimeValue[Cashflow[testFlow], .05, {2001, 1, 1}]

returns an NPV of .0256519, just as it should. And

FindRoot[TimeValue[Cashflow[testFlow], r, {2001, 1, 1}] == 0, {r, .05}]

gives us the IRR, in this case .0584316.

Mathematica clearly performs a compilation the first time my NPV is run, and while the native NPV and IRR computations are quicker than mine the first time one of them is run by about a factor of 50, the second time one of them is run (whether on the same data or not) my code is quicker than theirs by about 30%. Mine is also ecumenical about date formats, but in practice I find myself using their code anyway, as it is so concise.

The Wolfram Language is Here?

Stephen Wolfram released a formal introduction to The Wolfram Language two days ago but I am watching it only now.



The demonstration answers a question I had had since first hearing about this language — How is it different from Mathematica? The answer seems to be that this is the Mathematica programming language, IDE, and APIs, but conceptually separated from the Mathematica environment and given its own name. Thus, those of use who used to say we “programmed in Mathematica” will now say we “program in the Wolfram Language,” and the code we create in Mathematica can also be run in other environments (in web browsers, on embedded devices, etc.).

It’s nice to hear Wolfram credit this as a learning language; many people regard Mathematica as complex and obscure, but the reality is that for anybody who can handle leaving the world of imperative and object-oriented programming, it is extremely easy to learn, and to use and it’s extremely flexible. Separating the language from Mathematica may encourage novices to try it.

Wolfram doesn’t announce a release date in the video but I suspect it will be simultaneous with the release of Mathematica 10, which may be very soon.

Bloomberg to Mathematica

Several years ago, my team and I created a tool for importing data via the Bloomberg API directly into Mathematica. It works perfectly and we made both the code and the executable interface available via the Wolfram library. Apparently, Wolfram has removed it, perhaps because it competes with identical functionality in the Wolfram Finance Platform. But the Bloomberg to Mathematica link is free, and a Wolfram Finance Platform license costs $10,000, so people have asked me to put this online somewhere.

So here it is.
[2016 update — I’ve removed the old link as the executables and source code there were outdated. Get the current version at GitHub]

You need to have a Bloomberg Professional license, and as the Bloomberg data API works only on Windows, you need to be running Bloomberg and Mathematica on Windows (I’ve tested this in Windows XP through 8, and with versions of Mathematica through 9.0. Let me know if you run into problems with other setups).

The source code is included here; feel free to make modifications if you like. If you do anything interesting please share it back, it would be nice to see Mathematica used more widely in finance, and free sharing of tools like this would have to help.

Update, 2015:
I have added a new interface notebook for those using Mathematica 10. It fixes some bugs, increases flexibility, and most importantly, takes input and produces output using Mathematica’s new date and temporal data types, and using an Association rather than a list. It’s cleaner and more robust than the old method (though the old notebook will continue to work for those afraid of change.) If you upgrade from the old interface to the new one, note that all the functions have changed names, and their default installation path has as well. It’s all documented in the new notebook in the zip file above.

(Sample use of the old code shown below)

cpi2 = bbg2Mdates[
BBGetNF1["JNCPIYOY Index", "Px_Last", "19820101", "", "monthly"]];

tpxNorm2 =
BBGetNF1["TPX Index", "Px_Last", "19820101", "", "monthly"]]],
name -> "TPX"];

Show[DateListPlot[cpi2[[2]], Frame -> {True, True, False, False},
Joined -> True, Filling -> 0, FillingStyle -> {Blue, White},
GridLines -> False, PlotRange -> All, PlotStyle -> Purple],
DateListPlot[tpxNorm2[[2]], Frame -> {True, True, False, False},
Joined -> True, GridLines -> False, PlotStyle -> Pink],
PlotLabel -> "日本 inflation has been low since \!\(\*
StyleBox[\"before\",\nFontSlant->\"Italic\"]\) the market crashed",
Axes -> {True, False}, PlotRange -> All,
Epilog -> {(Text[
Style["stock market", FontFamily -> "Times New Roman",
FontSize -> 10, Pink], {AbsoluteTime["6/1/1986"],
4.4(*vertical position*)}, {-.01, 0}(*
how close to the end *)]), (Text[
Style["CPI", FontFamily -> "Times New Roman", FontSize -> 10,
Purple], {AbsoluteTime["1/1/1983"],
3.6(*vertical position*)}, {-.01, 0}(* how close to the end *)])}

japan inflation

A demonstration of the new code can be found at

[ Update March 7, 2016: New source code and executables are available on Github, at The Win32 binaries and instructions for use can be found at

If you want to load Bloomberg data into a version of Mathematica older than 10.0, contact me and I can send you an older release. ]

The most worthless subway stations in New York, part ii

In the last post, I demonstrated loading a GIS shapefile for New York City into Mathematica, loading the official subway station entry data from the city and overlaying it onto the map.

Below is the code to prepare and display an interactive version of the tool that also identifies and displays the stations on each line closest to each other and most remote.

preStationsForComp =
Drop[stations, 1][[
All, {3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 26, 25}]];
preStationsForComp[[All, 13]] = preStationsForComp[[All, 13]]/1000000.;
preStationsForComp[[All, 14]] = preStationsForComp[[All, 14]]/1000000.;

stationsForComp =
Union[Map[{#[[1]], #[[2]], #[[3]], #[[4]], #[[5]], #[[6]], #[[7]], #[[
8]], #[[9]], #[[10]], #[[11]], #[[12]], {#[[13]], #[[14]]}} &,
SameTest -> ((#1[[1]] == #2[[1]]) && (#1[[2]] == #2[[2]]) && (#1[[
3]] == #2[[3]]) && (#1[[4]] == #2[[4]]) && (#1[[5]] == #2[[
5]]) && (#1[[6]] == #2[[6]]) && (#1[[7]] == #2[[7]]) && (#1[[
8]] == #2[[8]]) && (#1[[9]] == #2[[9]]) && (#1[[10]] == #2[[
10]]) && (#1[[11]] == #2[[11]]) && (#1[[12]] == #2[[12]]) &)];
(* one randomly selected entrance per station/line *)

entranceFromLine[line_] :=
Select[stationsForComp, (#[[2]] == line || #[[3]] == line || #[[4]] ==
line || #[[5]] == line || #[[6]] == line || #[[7]] == line || #[[8]] ==
line || #[[9]] == line || #[[10]] == line || #[[11]] ==
line || #[[12]] == line) &][[All, {1, 13}]]

entranceFromSimplifiedData[line_, station_] :=
Select[stationsForComp, (#[[1]] ==
station && (#[[2]] == line || #[[3]] == line || #[[4]] ==
line || #[[5]] == line || #[[6]] == line || #[[7]] ==
line || #[[8]] == line || #[[9]] == line || #[[10]] ==
line || #[[11]] == line || #[[12]] == line)) &]

Module[{key, closestations, farstations, lineEntrances,
nearstationPair, farstationPair, closestToPointer},
key = Select[index, #[[2]] == whichline &][[1]][[1]];
closestations = First[screenedVitalDistances[[key]][[2]]];
farstations = Last[screenedVitalDistances[[key]][[2]]];
nearstationPair = {entranceFromSimplifiedData[whichline,
entranceFromSimplifiedData[whichline, closestations[[1]][[2]]][[
farstationPair = {entranceFromSimplifiedData[whichline,
entranceFromSimplifiedData[whichline, farstations[[1]][[2]]][[1]]};
lineEntrances = entranceFromLine[whichline];
closestToPointer =
First[SortBy[Map[{#, GeoDistance[#[[2]], p]} &, lineEntrances],
Graphics[{PointSize[Medium], Blue,
lineEntrances[[i]][[1]]], {i, 1, Length[lineEntrances]}]}],
Red, {Tooltip[Point[Last[farstationPair[[1]]]],
Orange, {Tooltip[Point[Last[nearstationPair[[1]]]],
Black, {Tooltip[Point[Last[closestToPointer[[1]]]],
PlotRange -> {{-74.05(*w*), -73.69(*e*)}, {40.54(*s*),
40.92(*n*)}}, ImageSize -> 340, ImagePadding -> 2],
farstations, {closestToPointer[[1]][[1]],
TableHeadings -> {{"closest pair", "farthest pair",
"closest to pointer"}, {"stations", "distance (m)"}}],
FontFamily -> "Times"]}, Center, 3]],
{{whichline, "F", "which line?"},
index[[All, 2]]}, {{p, {-73.87, 40.73}}, Locator}]

In our next installment, what are the most distant stations in the whole system?