In February this year I was contacted by the team at PACKT Publishing about being a technical reviewer for a book which was underway, in exchange I would get a free copy of the book and be listed in the book as one of the reviewers. I have toyed with the idea of writing a book for ages so this felt like a good idea to see what happens behind the scenes without committing to actually writing a book. I said yup to them, and today that book got published! You can get it at: http://bit.ly/json_cookbook (I am really excited about this)

The book* itself is a very interesting mix of content, from the very basics of JSON to introduction to MongoDB and storing data in it. It very much hits the nail on the head with the description “Quick answers to common problems”. What I think is really awesome about it, is that it really tries to cover a lot of languages & tools. So there is .NET, Java, Node.js, Android, ObjectiveC and more. The examples are really great as all work on a variety of OS’s too, so you can quickly try stuff out. I don’t think it is a book you It isn’t a book you would read end to end but rather give you guidance on where to start with problems related to JSON.

The experience itself of reviewing was interesting, each chapter took a few hours of reading and trying out the code and responding with details of issues found. The people I worked with at PACKT made it really pleasant. I never heard from the author or other reviewers, which in hindsight is odd but I think that maybe helped keep my responses focused and not have a bias in them.

If you do pick it up, let me know what you thought of it!

*Note: I’ve only see the content I reviewed, I haven’t seen the final book which maybe different.

Vorlon.js is an amazing tool for web developers as it brings the browser developer tools, aka F12, out of the limitations of a single browser and to the cloud. While you can run it on your machine, I believe to really get the full power of it, you need to put it on a dev server somewhere. The broader the reach, the more use. So with that mindset here is how to run it on an Ubuntu server and for this example I am use Azure to host that VM. I am using the Basic A2 (2 Cores, 3.5 GB) machine which is more than enough for Vorlon and means I can run the server pretty cheaply.

Step 1: Get NPM

Vorlon uses NPM, Node Package Manager, for distribution so you need that to get started. Before you begin, make sure you are up to date:

sudo apt-get update

Once that is done you can run the following to install NPM:

sudo apt-get install npm

If you didn’t follow my advice on doing the update first, or you ran into issues, you may need to run the command again with extra parameters:

Step 2: Get Node.js

Vorlon is built with Node.js, so you need to get that too. This took me surprisingly long to figure out, but thanks to this page I came right. The commands to run are:

curl --silent --location https://deb.nodesource.com/setup_0.12 | sudo bash -
sudo apt-get install --yes nodejs

Step 3: Get Vorlon

Now that the machine is setup correctly, you can get a copy of  Vorlon using NPM with the next command:

sudo npm i -g vorlon

Step 4: Run Vorlon

Running Vorlon is easy, just type: vorlon

For Windows users, make sure it is all lower case. It takes about 30secs and you will see the command output pop up

Once that shows up you can access it!

/usr/bin/env: node: No such file or directory

Every time I ran Vorlon originally I got the following error message: /usr/bin/env: node: No such file or directory

This is because I didn’t have Node.js installed. I had npm, which I assumed brought Node.js with it but it doesn’t and so Vorlon won’t run. In hindsight it is obvious, and even the error message tells me that but at the time – well it was an hour to figure out :/

Azure – don’t forget those endpoints

If you are on Azure, you will also need to setup the endpoints to allow access to the VM. To do this, go to the portal and open you VM. Click All Settings, then Endpoints, then Add. From there you need to add an endpoint that maps the public to the private. I keep both the same (1337) but you could make the public something different.

You want to code in TypeScript. You want to code with Sublime Text. Today, that is really really easy to do, thanks to Package Control and Microsoft, who is now providing a first class plug-in for Sublime Text to enable & light up TypeScript.

Step 1: Install Package Control

The installation instructions for it can be found here, but in summary you press Ctrl+` and then paste the following code (for Sublime Text 2, for 3 see their website) into the text box and press enter

import urllib2,os,hashlib; h = 'eb2297e1a458f27d836c04bb0cbaf282' + 'd0e7a3098092775ccb37ca9d6b2e4b7d'; pf = 'Package Control.sublime-package'; ipp = sublime.installed_packages_path(); os.makedirs( ipp ) if not os.path.exists(ipp) else None; urllib2.install_opener( urllib2.build_opener( urllib2.ProxyHandler()) ); by = urllib2.urlopen( 'http://packagecontrol.io/' + pf.replace(' ', '%20')).read(); dh = hashlib.sha256(by).hexdigest(); open( os.path.join( ipp, pf), 'wb' ).write(by) if dh == h else None; print('Error validating download (got %s instead of %s), please try manual install' % (dh, h) if dh != h else 'Please restart Sublime Text to finish installation')

It takes a few seconds where nothing happens and then, without much fanfare, the message window will say “Please restart Sublime Text to finish installation” – so do that.

Step 2: Add the TypeScript package

Press Ctrl+Shift+P to bring up the command palette and type in Package Control: Install Package and press enter.

You can use the arrow keys to navigate the list quicker and press enter on the right item rather than typing everything.

In the next command palette window type TypeScript and hit enter.

In the status bar it will start to download and install TypeScript:

Once that is done, it is done and you will get glorious TypeScript in Sublime!

Part 2 is here, click that now (it will open in another window) so you can continue the fun when you are finished here

The following image is a screen shot I took of a webpage in Chrome. If you view that same website, you will get a different experience because Telkom is using the same techniques as a man-in-the-middle attack to edit the code which the website uses, to serve Telkom’s own content.

The important bit which has been added, is the piece in the lower right hand corner of the page, which Telkom has added to the website without the permission of the website owner or the customer of the Telkom ISP account.

Admittedly this is a relatively benign addition, in fact it maybe seen as useful, and I can see it being sold that way to non-technical managers and executives. Do not be fooled though, even this simple addition can cause major issues for you.

Breaking of websites

It is impossible for Telkom to know what this addition will do to every website on the web. I will cover below the technicality of how it works below but in short they are adding JavaScript (the code of the web pages) to each page and that additional code could interfere with the existing code and web pages in unforeseen ways and ultimately can break a web page. The sheer size and complexity of the internet says that it is impossible for them to know for sure that they are not break a single website.

Exposing of security risks

The second issue is that they are exposing users to the potential of massive security risks. They may argue that their code is secure now and that maybe true, however that is a point in time view.

At the moment they are using a JavaScript library (a library being reusable code web devs use to make it easier to build websites) called jQuery and while the version they use has nothing reported regarding security issues, that doesn’t mean that in the future it will still be as secure. If a vulnerability is found in that library or in the additional code they wrote, it could be used to launch an attack against their users. You need to ask yourself, do you trust that the developers at Telkom who built this so much they could not have introduced a security issue? Do you trust that they are that good?

A second issue around the security, is that they need to host their content & systems which do this attack somewhere. If that server/s is compromised – either from an external attack or from a disgruntled employee they could use that for XSS attacks or key sniffing attack! As this “feature” becomes more widely known, you can be assured that those servers will become the target of more attacks because of their value of leveraging them to launching attacks on users will be very big. The reality is that it is just a matter of time before those servers are compromised.

Taking away your freedom to privacy

While it will always be possible for ISPs to see what the traffic you are sending & receiving is, it shouldn’t be easily accessible to the staff and people at the ISP. Having a server that can do this manipulation means that they are enabling a very easy point for someone to capture traffic and see what you are doing on the internet in such a way that it will be hidden from you.

While I am sure they will tell you they take security very seriously and that they do not allow that type of access to employees what is stopping an executive at a later stage from using this to prevent adverts from MTN showing up or causing web pages that support EFF or the DA to not load at all? Nothing and they have that power to do that, without oversight and without your permission. Do you trust Telkom enough to not abuse that?

Telkom doing this is violating a founding principal of the Internet: Net neutrality

Performance

Finally, and this is last because it is the least of the issues, they are making you download more 84.8Kb of extra code (see here for the data) and ~120Kb of extra images plus the manipulation of the web page slows down rendering. In short, they are making the web slower for you and helping use more of your bandwidth. They are also adding in extra servers and network hops which add even more performance concerns.

Technical details

How are they able to do this? From my testing they only do this to HTTP traffic, not HTTPS traffic, as that traffic is encrypted. While the encryption should help, it is not impossible to silently break and inject content as shown with the recent Lenono Superfish disaster which is basically the same thing as Telkom is doing except on an ISP level. It isn't easy or likely reasonable for them to do that, and I am not implying they are, just that they could and maybe motivated to add this to secure websites.

The first step in understanding this attack, is that Telkom is not changing the HTML (i.e. the web page) that is sent to your machine, that remains untouched (you can see the HTML for the above page at the time I got that injection of HTML here). So where is the new content coming from?  Telkom is very cleverly intercepting certain calls and redirecting them, so that unless you are actively looking for this, it appears transparent to the website and the user! What they are doing is watching for JavaScript files to be requested, and then appending additional code into those files. The additional code being jQuery 2.0.1 and then their own JavaScript which then is used to manipulate the web page.

To explain this it might help to know what a normal process of getting a file looks like:

1. You web browser requests a file that is needed for the website to load the web page
2. It hits the Telkom proxy, which is meant to help speed up the Internet by providing content to you from a closer location. They may not have a proxy, many ISPs do this so that they can provide a pure & secure experience.
3. If the proxy doesn’t have the content, it should ask the web server for it.
4. The web server provides the content to the proxy.
5. The proxy returns it to the browser unaltered.

The way this attack works is as follows:

1. You web browser requests a file that is needed for the website to load
2. It hits the Telkom proxy, which now checks to see if it is JavaScript or not that is requested.
3. If it is JavaScript, the request is changed to another server to handle.
4. That server then requests the original JavaScript from the website.
5. And the website provides it.
6. The Telkom server now appends its content to the JavaScript.
7. The resulting modified code is sent back to the proxy
8. The proxy returns the request to the browser.

Disclaimer: This is a rough view of it and without inside knowledge I cannot guarantee it is exactly like that. It maybe that there are changes but the principal will be the same.

If you sniff the traffic you can pick up the manipulation as the URLs for the JavaScript is being requested is being changed and you get URL’s that look like this:

http://196.25.211.41/n/50fc073bb429a4eb303a6420235e9381.js?q=Pb4KQL0xYJ…
http://196.25.211.41/n/50fc073bb429a4eb303a6420235e9381.js?q=RRNoU9x7uZ…

If we break down that URL, it is the server portion and a JavaScript file that Telkom generates [http://196.25.211.41/n/50fc073bb429a4eb303a6420235e9381.js]. The URL has two parameters, one a unique ID [q=Pb4KQL0xYJ%2F53kOn%2FSnvQA%3D%3D%7C1435397545%7C02dfd5fbe207b80c1859ed08333cd16d] to prevent caching (I assume) and the second the original file it is changing [url=http%3A%2F%2Fajax.googleapis.com%2Fajax%2Flibs%2Fjquery%2F1.7.1%2Fjquery.min.js].

Note: these URLs do not work outside the Telkom ADSL network.

FAQ

Are you sure your OS / Browser isn’t attacked? Are you sure this is Telkom?

In the image above it is Chrome on Windows 10. I have been able to reproduce this on my Windows Phone and also using Microsoft’s new Edge browser. This is on the network layer

And here is it again on the jQuery website using Microsoft Edge:

 

How are they showing content without changing the HTML?

JavaScript has the ability to edit and inject HTML. Here is the HTML for a page that is attacked and below is an image of the HTML which the browser has rendered. The differ because the HTML has been injected using JavaScript:

How can you be sure that this isn’t an agreement with the web site owner?

Because they are doing this everywhere. I was able to easily reproduce this with the programming site, StackOverflow (see image below). In this case they manipulated the jQuery JavaScript which that website uses. This is a great case because they load all their content from a CDN and you can compare what you got to the CDN and it shows a difference. StackOverflow will confirm they have never agreed to this.

Do you have examples of the JavaScript?

Yup. Here is the modified Modernizer from the PowerBase website: http://pastebin.com/1p4m5LnM

Here is the modified jQuery from StackOverflow: http://pastebin.com/XXY6b3n7 

I am on Telkom ADSL and I am not seeing this, you are wrong!

• You need to have Telkom as your ADSL & ISP. If I switch my ISP to FNB, for example, this doesn’t happen.
• Second it doesn’t happen on HTTPS, or at least I’ve never seen it. StackOverflow is a good example where it would always happen on HTTP, but when I switched to HTTPS (they support both) it stopped.
• It isn’t constant. I first picked this up a few days ago and then it stopped until the weekend.
• It also doesn’t happen on every website. I am assuming there is some requirements in regards to the JavaScript that they need to be able append to it.

What can I do to protect myself?

You can’t block those servers as that will break the websites where Telkom is injecting content into. Telkom has a web page which allows you to opt-out. I haven't been able to confirm it works yet but I will be keeping an eye out for it. Another possible tool to help protect you, you could use the HTTPS Everywhere extension to increase the amount of websites you access with HTTPS, thus lowering the amount of this type of nonsense.

Ultimately, you need to trust your ISP. They can see your data and manipulate it. You need to trust they won't. If they show they are unwilling to live up to the principals of the Internet or violate your trust, you only option is to change ISPs.

What if Telkom lets me opt-out, that is good enough right?

Opting out means that they will still have this system. The system will just check the database to see if you get the original files or the modified files. You still are slowed down for the check (versus performance of compared no check at all because this system is taken out). In the best case (i.e. just a DB lookup and then a redirect), once opted out that performance difference will be so marginal it doesn't matter. In the worst case, they could still direct you through the same (potentially vulnerable servers) and you could get a lot more overhead due to that, bad code etc....

The only approach which introduces no further security issues, no violation of net neutrality and no performance impact is for Telkom not to do this at all.

Updates

As I am totally happy to admit when I am wrong, any updates will be listed here

12h05 10 July 2015 - I wrote many ISPs do not use proxies by mistake. I meant many do, but only for performance reasons. Not for changing of content and added more info & clarification on the HTTPS breaking.

15h00 29 June 2015 - it was pointed out to me it isn't just the JS you download, it is the MASSIVE image too. So rather than 80ish Kb it is almost 200Kb in total! I've updated the article to reflect that.

15h30 29 June 2015 - Added points reflecting that this is a violation of net neutrality and the HTTPS everywhere extension which might help some people.

18h25 20 June 2015 - Added information about opt-out option. Added information around performance and how it may or maybe impacted by opt-in out. All of that had some rewording to the "What can I do to protect myself?" section.

I try not to rant on my blog and mostly I succeed. Today though I wanted to complain to Telkom about them inserting content into web pages on Twitter so I head to their website to find their Twitter account and I don’t find one, I find THREE different Twitter accounts and a situation so bad I cannot keep shut!

It isn’t ideal to have three and I can understand different components need it, in fact they look like a business account (1), their cellphone brand (2) and a general one (3). The part that baffles is what happens when I checked them.

The business account

This account has been taken over by spammers :facepalm:

The mobile account

That account is suspended :facepalm:

In fact only the general account works! Shit happens, people mess up and accounts get suspended or lost – it isn’t ideal or professional and you can mitigate the damage by removing it from your website. The fact Telkom can’t/won’t speaks volumes to their inability to care. There is no acceptable reason for those bad links to be up there and who ever runs their website should hang up their HTML spurs and find a new career.

Let them know

Want to help let them know this isn’t acceptable? Here is the form for the webmaster and here is a handy link to Tweet to them quickly.

Yesterday in Johannesburg we had an amazing //Build Tour event! We had top international speakers like Pete Brown join us and wow the audience with Windows 10, IoT, Edge and more! Unfortunately for the international speakers, they had flight issues and we had a hour and half gap before we could kick off, so a bunch of the local Microsoft people got together to provide a bit of local flavour in a pre-keynote event. I was very excited to share about Visual Studio 2015 and it got cut at the last minute as the presenters arrived! Part of my prep was to create a video of my talk and you can watch this content now:

You can get the demo code on GitHub.

With an agile team you should meet daily for a quick meeting. Normally that takes the form of each person answering a three question status update (What did I achieve, what will I achieve, what is blocking me). For a team working together it means these steps:

1. Team arrives at the shared team board
2. Team goes round and gives status update
3. Team goes back to work

Just because your team is distributed doesn’t mean this shouldn’t happen – it just means that you’ll find a few extra steps:

1. Team members who work together go to booked meeting room (you need a meeting room so those remote can hear over the normal work environment noise)
2. Be surprised to find some one in the room
3. Explain that you booked the room
4. Wait for them to leave the room
5. Connect the projector
6. Realise you left your microphone in your laptop bag, return to desk get it, return to meeting room
7. Plug it in
8. Open the digital scrum board
9. Start meeting software and join meeting
10. See one remote team member is already there – apologise for being late
11. wait for response
12. wait for response
13. realise they can’t hear you.
14. Both the remote team member and yourself try to figure out what is not working
15. Figure out that there is a mute hardware button that today (and today only) is turned itself on
16. Try again – success, they can hear you
17. Wait for the product owner to join.
18. Give up waiting and kick off meeting (everyone is seated now cause no ones legs last that long)
19. Go around the team and give status update
20. Product owner joins during second last team members status update
21. Product owner starts sharing a bunch of bugs and task updates that someone else is now capturing live
22. Finish status updates
23. Team decides to use this time that we are all here to discuss technical info around the new stuff from the product owner
24. Product owner is late for meeting and has to disconnect
25. Someone walks into the meeting room and asks if this is the meeting for organising the year end party
26. silence
27. No one answers
28. silence
29. Scrum master answers
30. Team discusses technical issues. Half are marked as needing more info and will be discussed with product owner in tomorrows meeting
31. Call ends
32. Team goes to get coffee (they need it) and says how glad they are using agile cause they save so much time with less meetings
33. two hours later
34. other team member joins call alone because the time zone he is in changed to daylight savings today

Repeat daily

Community night is an initiative from Microsoft South Africa to help everyone have a platform to engage and grow, through learning and networking in board communities. That sounds awesome on a press release, but what is it?!  It means that on the second Tuesday of the month, Microsoft in Johannesburg opens it’s meeting rooms up for user groups to use to meet, basically giving a safe venue and infrastructure that groups need for face to face events. Microsoft also sponsors food and drinks for the groups so that you can have those amazing hallway conversations. There are a few groups that take part in community night, but it does change based on the groups – so here are some of the regulars:

• Developer - http://developerug.org.za/
• Information Worker - https://www.facebook.com/InformationWorkerZA
• GameDev - http://www.makegamessa.com/
• Xamarin User’s Group - http://xamarin.meetup.com/
• SQL - http://sqlusergroup.co.za/
• Lean StartUp Circle - http://www.meetup.com/Lean-Startup-Circle-Joburg/

if you are interested in getting your group involved, please reach out and I will help put you in touch with the right folks.

Sometimes I get a PDF I want to print, but there is a background image or colour which will eat too much ink & I would like to remove it. The example I am going to use is the rules reference for Tales of Arabian Nights from The Esoteric Order of Gamers, which has a very bold parchment background.

This is a fairly simple job, you just need the right tools and that tool is the FREE (and open source) Inkscape. Once you have it downloaded and installed, browse to your file and open it – being a PDF you will be presented with a dialog asking you to convert it. You need to do a page at a time, so select the page at the top.

 

Once you hit OK, it will import it and you can then select the background image and delete it. You’ll note that some of the text is the wrong font & that is because I do not have the right font installed. Often you will have clean up like this, and adding back logos, to do but that is up to you in regards to how much you want to polish it.

Once done user File –> Save As to save back to a PDF. The biggest pain is that you need to do this page by page and in the end you’ll have numerous files (one for each page), but since the goal is to print it – it isn’t a big pain.

Here is the before and after for another one I did:

Someone who attends the awesome developer user group (DUG) is: 26-35, a senior C# developer, who is not looking for a new job and is working on websites or mobile apps at company of 10 people or less. Statistically speaking. This is what came out in the recent DUG annual survey. This was the second time we ran the survey (2014 can be found here) and we tried to keep the questions similar so we can compare. The goal of the survey is ultimately to understand the audience better so that we can give the audience a better event.

Satisfaction

For us, as the leads of the group the  sat numbers are very important, and we are very happy to see a small uptick in these numbers.

As for complexity of sessions that is still near the middle but is higher, meaning they are slightly more complex than last year.

Start Time

Going into last years survey the start time for Johannesburg was a big discussion point, what time should the group start at and last year 49% of people wanted it to remain where it was, 6% earlier and the remaining 45% later. This year that has changed with 57% of people wanting to keep it as is, no one wanting it earlier and 43% wanting it later.

This year we have Pretoria data too, and it is a very different picture with only 37% happy with the current time and 63% wanting an earlier time. There is no clear winner with an earlier time as 16h30, 17h30 & 18h00 all got the same number of votes. This group will be shifting earlier going forward so hopefully we see an improvement next year.

Why do you attend?

Attendance is largely to skill up and network. For the most part these numbers are similar to last year. The only interesting changes are around employment those wanting a job is lower (down 6% since last year) while those looking is higher (up around 5%).

Demographics

Breaking down the details of the people at the group, starting with age:

• 18 – 25:    10%
• 26 – 35:    58%
• 36 – 45:    26%
• 46 – 55:    4%
• 56 – 65 :   2%

Skill level:

• Senior    67%
• Intermediate    18%
• Junior    14%

We have seen a increase in juniors over 2014, which is a great trend!

Languages:

Compared to 2014, the losers are

• C# – down 3%
• C++ – down 1%
• Python, PHP, Scala, SQL – all losing 1% and dropping away completely

The winners are:

• Java – up 1%
• JavaScript – up 9%
• Objective C – up 5%
• VB – up 1%
• Ruby, Haskell and Meteor – all new at 2% this year

What work do we do?

There is a trend away from web technology, with internal websites losing 8% and public facing websites losing 7%. Mobile dev is up 9%.

Company size:

• 1-10    28%
• 11-20    12%
• 91-200    8%
• 41-80    8%
• 21-40    2%
• >200    0%

Role in purchasing:

• I can suggest/influence purchases    40%
• None    42%
• I make the final say    18%

Comments

The majority of the comment were positive and supportive of us - thank you so much! There was one trend that popped out around video recording the sessions. This is something we are investigating and we do want to offer the videos and we want to make sure we have a good reliable system for that. This is not an easy problem to solve but we are working on it.

Raw data

If you would like to play with the data yourself you can get it below.