Smarter Screen

I spend a lot of time in the kitchen, I love to cook and so I am often in there with my phone listening to a podcast or, if it is a Saturday morning, watching Show of the Week. I am not alone in this behavior, everyone in my home does this and often at dinner we share YouTube videos by propping a phone up on the toaster and huddling around it. This was clearly time to improve the experience with a kitchen screen—a smart TV would be perfect, but their lack of support means it is a showstopper for me... so I put together a smarter screen.

Build List

Powering this is a Raspberry Pi 4. I grabbed the 4GB model, just because... I don't have a smart reason for that decision. If you're in South Africa, I grabbed mine from PiShop and grabbed the essentials kit. Putting together the "case" was maybe the most head-scratching aspect since it's just screws, plastic, and a fan—no instructions. Also ordered from PiShop was the remote control since I want this to be like a TV. I do not want a keyboard or mouse. I opted for the OSMC Remote Control, which has a small USB dongle and uses radio signals rather than infrared, meaning it doesn't need line of sight. Since the Pi will be behind the screen, line of sight would be an issue. The remote "just worked," which was so awesome.

For the screen, I ordered a LG 24MK400H, which was perfect for my needs—wall-mountable and on special 😄. The mounting solution: I grabbed a Brateck LDA18-220 Aluminum Articulating Wall Mount Caravan Bracket, which was really awesome and easy for mounting. This came with instructions, but they were poor—experimenting first helped me find a happier setup.

With all of that, I had everything I needed to get it running.

OS

The Pi kit came with a MicroSD card with NOOBS preinstalled, and all I needed to do was, when booting, hold Shift and select the LibreELEC OS to install. LibreElec is a really basic OS, which is "just enough" to run the Kodi media center software. Going through the setup got it up and running within about 30 minutes.

Add-ons

I don’t have a "library" of media; I just stream the content I want, so installing the add-ons I needed was key. Here’s what I went with:

Notes

The only strange part of the setup was that each time Kodi booted, I got a prompt saying there was an update for LibreElec... but the settings for LibreElec had nothing in it for the update and no way to do it. Thanks to Reddit, I was able to switch it to manual and update the setup and then switch it back.

Go to Settings > LibreELEC > System. Change automatic updates to manual (I'm not even sure if auto-update works at all—I've had it set to that before, and it never auto-updated). Change the update channel to LibreELEC-8.0. Select available versions and pick the newest one (8.2.3 at the time of writing this).

If this was how you were trying to update, I'm not sure. I would say back up your LibreElec install and then start fresh with a new version.


DevFest 2019

Today I was honored to be part of the second DevFest in SA with a talk about Kotlin, Micronaut, DataStore, and other fun tech—but more on how we ended up where we are with our current project. It was a tech lead doing a retrospective with tech sprinkles to get everyone involved. If you want the code, it’s on GitHub, and the slides are below:


When the world sees a 500... but the server promises it is a 200

Here is the story of all the work I did this week, and it is so odd I feel it needs to be shared... but let’s talk about the world the problem can be found in. # The world It is a µservice (that’s the ohh, look how smart I am to use a symbol for micro) written with Dropwizard and deployed in a Docker container, with Traefik in front of that. To hit it, you go through an ingress controller and a load balancer.

+---------------+
|               |
|   Internet    |
|               |
+---------------+
        |
+---------------+
|               |
| Load Balancer |
|               |
+---------------+
        |
+---------------+
|               |
|   Ingress     |
|               |
+---------------+
        |
+---------------+
|               |
|   Traefik     |
|               |
+---------------+
        |
+---------------+
|               |
| Microservice  |
|               |
+---------------+

The microservice acts as a BFF (backend for frontend), so it does some auth fun and makes calls to an internal API, manipulating them (e.g., changes the data structure). We have a number of different REST-style calls across GET, POST, PUT, and DELETE.

In terms of environments, we obviously have a production environment and an integration environment, which are set up the same way. We have a stubs environment where we fake out the internal API. Lastly, we can run the microservice on our laptops—but there’s just the microservice, no Traefik, etc. # The Problem

When we run our load test from outside, everything works except one call, which fails 98% of the time with a HTTP 500. Other calls (even the same method) all work. Load tests run against the stubs environment, and the same call works perfectly on our laptops, in integration, and in production. We can even run the fake internal API on the laptops with the load tests, and it works fine there. Basically, one call fails most of the time in one environment... 🤔

Grabbing logs

When we pull the logs for the microservice in stubs, things get weirder—it is returning an HTTP 200 😐. This is the same experience we get everywhere else; it works... except in stubs, where the load tests get a 500.

+---------------+
|       500     |
|               |
|   Internet    |
|               |
+-------+-------+
        /
+-------+-------+
|       200     |
|               |
| Microservice  |
|               |
+---------------+

Further log pulling shows 500s in the load balancer and the ingress, so somewhere between the microservice and Traefik, the HTTP 200 becomes a HTTP 500... but we don’t have logs for Traefik we can pull, which makes this a bit harder to determine. # Logging onto Traefik

Next, we logged onto Traefik and decided to curl the microservice directly—and lo and behold... we get a 500 🤯. And to make it more interesting, the microservice logs still show it is returning a 200—like what the actual?!. Could there be a network issue or magic?

Interestingly, the 500 came with an error saying: "insufficient content written."

Insufficient content written

This led me to looking at the content and what we’re sending. I noticed we are sending Content-Length and the body, and guess what... the length of the body does not equal the Content-Length... oh 💩. This is a client-side HTTP error, where the server sends an incorrect amount of body, so the client goes: "Well, the server must be wrong" and raises a 500.

I’ve always thought 500 errors were server errors and thus could only be raised on the server.

The fix

The fix was simple: in our server, we were using Response.fromResponse to map the internal API to the public API. It was copying the Content-Length from the internal API and sending it along. The fix was to delete the Content-Length header before calling fromResponse, ensuring it would rebuild the header and be correct.

The reason why it didn’t fail elsewhere: the version of the mock API we used added Content-Length, but newer versions and the real APIs used chunked encoding, which never set the header, so there’s no issue there.

This was a long road to understand the issue, and one line to fix it—and totally a new experience in learning that server errors can occur client-side too.


VSCode + Catalina

For the most part, the initial upgrade to macOS Catalina was uneventful; I was caught unaware by the wave of permission requests that greeted me—it was 2 minutes of clicking accept or deny and continuing on with my day (though, how normal users will cope is beyond me... it feels very un-Apple). The two issues I did run into were the need to reconfigure Google Drive (again, a minor 2-minute activity) and trying to get VSCode to work properly. This was a lot more annoying. The initial issue was that Git could not be found—this broke all of source control in VSCode. The fix was to run the following from the terminal:

xcode-select --install

and then restarting VSCode. Once that was fixed, the next issue was that I could no longer sign commits with GPG, which gave a similar issue to the initial Git not being found. The correct fix for VSCode is to add it to the list of Developer Tools, which you can find under Security & Privacy. Once a restart of VSCode was done, everything just worked. I also added Terminal to the list, which stopped autocomplete from Fish Shell from constantly prompting too.


important reminder that diversity is important

(I tweeted this story today, and some wanted to be able to share it, so here it is.) Everyone gets along well at work, except Jim. He never comes to after-work drinks. He doesn’t go to get Friday burger lunch with the team. He doesn’t chip in for Sarah’s farewell gift. When he gets “unicorned” (i.e., he left his machine unlocked, and someone emailed everyone), he doesn’t buy the punishment cake. This annoys everyone. Why doesn’t he just play along? It’s harmless fun. Jim sucks… Right? He just isn’t trying to be part of the team… Right?

Well, Jim can’t afford the transport from after-hours drinks or the expensive burgers or the cake. He would like to. Financial diversity is a hard thing to spot in a team, but it’s a real thing. Are you excluding people because they aren’t privileged like you?


Learning Kotlin: The notnull delegate and lateinit

Note This is the 30th post in a multipart series. If you want to read more, see our series index. This follows the introduction to the by operator and delegates.

A lot of introductions to Kotlin start with how null is opt-in because that makes it safer—we even looked at this way back in post 8. The problem, though, is that sometimes we don’t know what we want the value to be when we initialize it and we know that when we use it we don’t want to worry about nulls. The ability to change from null to non-null might seem impossible, but Kotlin has two ways to do the impossible. Before we look at the two solutions, let’s examine a non-working example.

Here we have a class to represent a physical building, and we want to store the number of floors, which we get from calling an API (not shown). The problem is there’s no good default value: a house with just a ground floor is 0, a multi-story office building could have 10 floors, and an underground bunker could have -30 floors.

class Building() {
    var numberOfFloors: Int

    fun getBuildingInfo(erfNumber: String) {
        // call municipality web service to get details
        numberOfFloors = 0; // 0 causes 0 = G like in elevators
    }
}

fun main(args: Array<String>) {
    val house = Building()
    house.getBuildingInfo("123")
    println("My house, up the road, has ${house.numberOfFloors}")
}

This won’t compile as it states: Property must be initialized or be abstract for the numberOfFloors field.

notnull

Following on from our previous delegates, we have the notNull delegate (import: kotlin.properties.Delegates) to allow the property to not be set initially and then enforce initialization later.

import kotlin.properties.Delegates

class Building() {
    var numberOfFloors: Int by Delegates.notNull()

    fun getBuildingInfo(erfNumber: String) {
        // call municipality web service to get details
        this.numberOfFloors = 10;
    }
}

fun main(args: Array<String>) {
    val house = Building()
    house.getBuildingInfo("123")
    println("My house, down the street, has ${house.numberOfFloors}")
}

This example will print that we have 10 floors. If we comment out line 14, we’d get the following exception:

Exception in thread "main" java.lang.IllegalStateException: Property numberOfFloors should be initialized before get.

So not exactly a null exception, but close enough—it achieves the same goal.

lateinit

Another option is lateinit, which we can add before the var keyword—but we can’t use it with primitive types like Int or other primitive types. Instead, we need to convert it to a class. This is a nice and simple solution.

data class Floors(val aboveGround: Int, val belowGround: Int)

class Building() {
    lateinit var numberOfFloors: Floors

    fun getBuildingInfo(erfNumber: String) {
        // call municipality web service to get details
        this.numberOfFloors = Floors(2, 40);
    }
}

fun main(args: Array<String>) {
    val house = Building()
    house.getBuildingInfo("123")
    println("My house, in the middle of the avenue, has ${house.numberOfFloors}")
}

Once again, if we comment out line 14, we get an exception as expected:

Exception in thread "main" kotlin.UninitializedPropertyAccessException: lateinit property numberOfFloors has not been initialized.

lateinit vs. notnull

As we can see in our simple examples, both achieve the goal—but they each have their own limitations and advantages:

  • notNull being a delegate requires an extra object instance for each property, so there’s a small additional memory/performance overhead.
  • The notNull delegate hides the getting and setting logic in a separate instance, which means anything that works with the field (e.g., Java DI tools) won’t work with it.
  • lateinit doesn’t work with primitive types (Int, Char, Boolean, etc.), as we saw above.
  • lateinit only works with var and not val.
  • When using lateinit, you gain access to the .isInitialized method, which you can use to check initialization status.

Learning Kotlin: The vetoable delegate

Note This is the 29th post in a multipart series. If you want to read more, see our series index or this follow-up on the introduction to the by operator and delegates.

The vetoable delegate is usable with variables only (i.e., you must be able to change the value), and it takes a lambda that returns a boolean. If the return value is true, the value is set; otherwise, it is not set.

For example, we’ll ensure a User's age can never decrease—since time only moves forward:

import kotlin.properties.Delegates

class User() {
    var age: Int by Delegates.vetoable(0) {
        property, oldValue, newValue ->
        newValue > oldValue
    }
}

fun main(args: Array<String>) {
    val user = User()
    user.age = 10
    println(user.age) // 10
    user.age = 15
    println(user.age) // 15
    user.age = 10
    println(user.age) // 15
}

Learning Kotlin: The map delegate

Note *This is the 28th post in a multipart series. If you want to read more, see our series index *This follows the introduction to the by operator and delegates.

The next delegate is actually built into something we’ve looked at before—map—and if you use by to refer to one when you get the value, it looks up the map’s values. As a way to illustrate this, we’ll take some JSON data and parse it with the kotlinx.serialization code. We then loop over each item and convert it into a map, which we use to create an instance of User.

import kotlinx.serialization.json.*
class User(userValues: Map<String, String>) {
    val name: String by userValues
    val eyeColour: String by userValues
    val age: String by userValues
}

fun main(args: Array<String>) {
    val json = """[{ "name":"Robert", "eyeColour":"Green", "age":36 },{ "name":"Frank", "eyeColour":"Blue" }]"""
    (JsonTreeParser(json).read() as JsonArray)
        .map { val person = it as JsonObject
               person.keys.associateBy({ it }, { person.getPrimitive(it).content })
        }.map { User(it) }
        .forEach { println("${it.name} with ${it.eyeColour} eyes") }
}

This code outputs:

Robert with Green eyes
Frank with Blue eyes

However, if we change line 25 to include the age, we get a better view of what’s happening:

}.forEach { println("${it.name} with ${it.eyeColour} eyes and is ${it.age} years old") }

This simple adjustment changes the output to:

Robert with Green eyes and is 36 years old
Exception in thread "main"
java.util.NoSuchElementException: Key 'age' is missing in the map.
        at kotlin.collections.MapsKt__MapWithDefaultKt.getOrImplicitDefaultNullable(MapWithDefault.kt:24)
        at sadev.User.getAge(blog.kt)
        at sadev.BlogKt.main(blog.kt:27)

As you can see, the map isn’t initializing the values ahead of time. Instead, it’s merely used to look up a value in the map when the property’s getter is called.


Learning Kotlin: The Observable Delegate (with a slight detour on reference functions)

Note * This is the 27th post in a multipart series. If you want to read more, see our series index * Koans used here are 34 and 35

Following on from our introduction to the by operator and delegate properties, this post looks at the second of the five built-in delegates, observable. This delegate allows you to intercept all attempts to set the value. You can achieve this with a setter, but remember you would need to duplicate that setter code every time. With observable, you can build the logic once and reuse it over and over again.

Once again, let’s start with how we would do this without the delegated property:

class User() {
    var name: String = ""
        set(value) { DataChanged("name", name, value) }

    var eyeColour: String = ""
        set(value) { DataChanged("eyeColour", name, value) }

    fun DataChanged(propertyName: String, oldValue: String, newValue: String) {
        println("$propertyName changed! $oldValue -> $newValue")
    }
}

fun main(args: Array<String>) {
    val user = User()
    user.name = "Robert"
    user.eyeColour = "Green"
}

Note: We need to manually define the setter twice, and in each, we must change a value—something you’ll likely overlook when copy-pasting the code.

In the next example, we switch to the observable delegate, which lets us call the same function easily. While I don’t recommend this for production, I did demonstrate two different ways of using it in this example.

For age, since the second parameter is a lambda, I created one and passed the parameters to my function—this is how most demos show usage. For name, however, because my function has the same signature as the lambda, I could pass it directly to observable, which I find much nicer. However, since we need to pass a reference to our function, we must prefix it with ::.

package sadev

import kotlin.properties.Delegates
import kotlin.reflect.KProperty

class User() {
    var name: String by Delegates.observable("", ::DataChanged)
    var eyeColour: String by Delegates.observable("") { property, old, new ->
        DataChanged(property, old, new)
    }

    fun DataChanged(property: KProperty, oldValue: String, newValue: String) {
        println("${property.name} changed! $oldValue -> $newValue")
    }
}

fun main(args: Array<String>) {
    val user = User()
    user.name = "Robert"
    user.eyeColour = "Green"
}

Learning Kotlin: The Lazy Delegate

Note This is the 26th post in a multipart series. If you want to read more, see our series index. The koans used here are 34 and 35.

Following on from our introduction to the by operator and delegates, this post looks at the first of the five built-in delegates, lazy. Lazy property evaluation allows you to set the initial value of a property the first time you try to get it. This is really useful for scenarios where there’s a high cost of getting the data. For example, if you want to set the value of a username that requires a call to a microservice—but since you don’t always need the username—you can use this to initialize it when you try to retrieve it the first time.

Setting up the context for the example, let’s look at two ways you might do this now:

  1. First way: You call the service in the constructor, so you take the performance hit immediately—regardless of whether you ever need it. It’s nice and clean though.

    class User(val id: Int) {
        val name: String
        constructor(id: Int) {
            // load service ...
            // super expensive
            Thread.sleep(1000)
            name = "Robert"
        }
    }
    
    fun main(args: Array<String>) {
        val user = User(1)
        println(user.name)
    }
    
  2. Second way: You add a load function, so you need to call that to load the data. This avoids the performance hit but is less clean—and if you forget to call load, your code breaks. You might think, “That will never happen to me…” I just did that right now while writing the sample code. It took me less than 2 minutes to forget I needed it. 🙈 Another pain: I need to make my name property mutable (var) since it’ll be assigned later.

    class User(val id: Int) {
        var name: String = ""
        fun load() {
            // load service ...
            // super expensive
            Thread.sleep(1000)
            name = "Robert"
        }
    }
    
    fun main(args: Array<String>) {
        val user = User(1)
        user.load()
        println(user.name)
    }
    

The solution to this is obviously lazy—it gives us the best of all of the above (clean code, no performance hit unless we need it, can use val, and no forgotten calls) with none of the downsides.

class User(val id: Int) {
    val name: String by lazy {
        // load service ...
        // super expensive
        Thread.sleep(1000)
        "Robert"
    }
}

fun main(args: Array<String>) {
    val user = User(1)
    println(user.name)
}