Tuesday, 30 October 2018

#2018 Trends



It's almost November but feels like a winter, I've tried to create cheat sheet with been trendy buzzwords/streams this year I had to face or work with. Did it for myself to read/refresh books/blogs during the upcoming vacation but might somebody will find it useful to be shared:
  • Java from Oracle isn't free anymore
  • Java as a language is lagging behind all others, Lombok and Spring Roo aren't demanded solutions. I still bet to Scala
  • Kotlin got popularity, finally 1.3 has been released
  • Rust 1.3 has been released
  • DevSecOps over DevOps, it's mandatory to invest into security
  • Prometheus over Graphite
  • Microservices is still the buzzword
  • Docker vs Podman, project CRI-O, OCID (Open Container Initiative Daemon) - well Ops in nova days is something that is quickly changing
  • IBM got Redhat, are there any other open-source independent whales?
  • Clouds becoming private for intermediate size companied
  • Ansible over Chef and Puppet
  • Python is great again
  • AI buzzword over data analysis (k-nearest neighbors classifier is accepted as magic spell)
  • Data Scientist and Data Engineer are demanded, to get outside of Hadoop, Spark, Kudu, Impala, Kafka, Ignite, Geode etc is a separate profession
  • Engineering Culture over Agile to address Conway's law
  • Vue, React are gaining popularity
  •  Kafka as a glue for everything. Got the competitor nats.io streams but for streaming it's still astable (by the way implemented in Go)
  • Yaml is for configuration
  • GraphQL is preferred to REST
  • Cassandra is preferred to MongoDB
  • GraalVM: is too a day before the fair to start looking into
  • JVM: still there is a demand for green threads

Tuesday, 9 October 2018

Scala Сheat sheet: Context Bound of multi typed kind

Because of rarely using this functionality every time has to find out how to use multi type for context bound.

For example there is class extending Function1:

abstract class Mapping[In, Out] extends (In => Out)

If we want to use Mapping[In, String] as a Context Bound for other class we should use type Lambda (type-level Lambda):

abstract class StringRequestResponse[In: ({type M[x] = Mapping[x, String]})#M]

In case of return should be parametrised as well:

abstract class RequestResponse[In: ({type M[x] = Mapping[x, Out]})#M, Out]

Revise Kotlin 1.3 from Scala experience perspective



Thanks to Google Kotlin is producing more and more informational noise. It's interesting to make an assessment spending less than hour to find out where the language is now and comparing it with Scala's experience. I'm not going to review feature by feature - but will try to get impression in offbeat way - choosing the most interesting/unaccustomed feature in 1.3 and trying to assist language's way of development comparing with pseudocode in Scala.

Let's look into upcoming release 1.3.  Kotlin Contracts is looking interesting and probably the biggest KEEP change in the release. Looks like something language specific (haven't seen it in other languages) that improves language itself and may bring some light to the Kotlin's "way of thinking".

Let's run through the proposal to and try to understand what it improves and why.
The first example/motivator is:
fun test() {
val x: Int
run {
// Next line is reported as 'val reassignment', because compiler
// can't guarantee that lambda won't be invoked more than once
x = 42
}
// Next line is reported as 'uninitialized variable', because compiler
// can't guarantee that lambda will be invoked at least once
println(x)
}
view raw ex1.kt hosted with ❤ by GitHub

Well, it's extremely hard to get why should we write the code like that. There is some background on how Kotlin implements calls of closure blocks (there was similar SIP-21 - SPORES in Scala, but  didn't gain popularity), skipping that - for particular example it feels more natural to use functional approach:
fun test(cond: Boolean) {
val x: Int = run { 42 }
println(x)
}
view raw sol1.kt hosted with ❤ by GitHub
I feel like Kotlin tries to make this code valid:
public class Example {
private final int state;
public Example() {
try {
state = 5;
} catch (Exception e) {
state = -1;
}
}
}
view raw ugly.java hosted with ❤ by GitHub
Kotlin doesn't require us to define val and initialise it immediately, but I don't feel it's nicer and better readable. There is an extra price - method run has a

contract {
    callsInPlace(block, InvocationKind.EXACTLY_ONCE)
}

The next one example is:

fun test(x: String?) {
require(x != null)
// Compiler doesn't know that if call to 'require' has finished successfully,
// then its argument, namely, 'x != null', is true
// Hence, there is no smartcast on the next line, while it is actually safe:
println(x.length)
}
view raw ex2.kt hosted with ❤ by GitHub

As for developer with Scala experience it's hard to understand the problem's domain - but keeping in mind that Kotlin is providing safety for null references problems - there is some sugar for the cases when safety is already checked against the reference and it follows some rules (not null in the example). Looks fine and it's imperative alternative to Monad's approach (will talk about this later). But if it were the production code I would prefer to avoid throwing the exceptions and separate execution of the side effect (println). Same as in previous example there is small complexity via introducing:
@kotlin.internal.InlineOnly
public inline fun require(value: Boolean): Unit {
contract {
returns() implies value
}
...
view raw ex2.kt hosted with ❤ by GitHub

Next one example is:
fun isString(x: Any?): Boolean = x is String
fun foo(y: Any?) {
if (isString(y)) {
// 1. Here, compiler has observed that 'isString(y)' returned 'true'
// 2. Looking at the contract of 'isString', compiler concludes
// that 'y is String'
// 3. This allows otherwise impossible smartcast on the next line:
println(y.length)
}
}
view raw ex3.kt hosted with ❤ by GitHub

Looks like pattern matching customisation - I had intuitive feeling that it's the way to handle the Union Types but it isn't. As for the guy without commercial experience with Kotlin - all the examples are too artificial to evaluate the syntax sugar coming with contracts. For example the last example looks much better if the pattern matching is applied
fun foo(y: Any?) {
when (y) {
is String -> println(y.length)
}
}
view raw sol2.kt hosted with ❤ by GitHub

Still it could be covered via Option/Either Monads or if there are more possibilities - better to look into Coproduct solutions. All the other examples are following the same paradigm if it were the real product's source code I would prefer to use best practices from Lambda and Categories patterns. Especially that handling nullable (empty) values has the same importance as validation/parallel validation or applying side effects in a good way. There is probably implicit advantage of Kotlin's sugar - allocate less memory and it would be great - but as we will see as soon as we require some feature like ?.let the extra memory allocation is inevitable.



Summary
Overall impression is quite positive. Kotlin is inventing some alternative to Lambda + Category solutions for specific problems.  Nullable reference is really weird solution that appears to be the centrum of all the problems/improvements in the given examples. Fortunately I haven't met NullpointerException quite some time in Scala - but should admit that one of the most used Monads is Option. Another  positive impression is: it's possible to write code in Kotlin from the first minute.

Bonus: Weird nullable types
The Null Safety is looking too noisy but we will play around this area comparing with Scala. Let's imagine we need to read (from property file)  three optional variables and build optional url object based on those. Pretty easy approach checkin all the 3 variables against null and creating the object, Scala allows to write something like:
def url(protocol: Option[String],
host: Option[String],
file: Option[String]): Option[URL] =
for {
p <- protocol
h <- host
f <- file
} yield new URL(p, h, f)
view raw play1.scala hosted with ❤ by GitHub

Or not using the for-comprehensions but Applicative's mapN:
def url(protocol: Option[String],
host: Option[String],
file: Option[String]): Option[URL] =
(protocol, host, file).mapN(new URL(_, _, _))
view raw play2.scala hosted with ❤ by GitHub

Let's find out can we do something similar. Kotlin doesn't support for comprehensions out of the box. Starting with brute-force solution for 2 params:
val p1: String? = null
val p2: String? = "p2"
assertNull(concat2(p1, p2))
/* Almost ternary operator */
fun concat2(arg1: String?, arg2: String?): String? {
return if (arg1 != null && arg2 != null) arg1 + arg2 else null
}
view raw play3.kt hosted with ❤ by GitHub

Let's use ?.let method:
assertNull(concat3(p1, p2))
/* Welcome to primitive lambda calculus or http://callbackhell.com/ */
fun concat3(arg1: String?, arg2: String?): String? {
return arg1?.let { a1 -> arg2?.let { a2 -> a1 + a2 } }
}
view raw play4.kt hosted with ❤ by GitHub

Looks fine for the case with two params - but wouldn't for more. Let's try to play with Monad's flatMap and unit:
/* From Scala
trait M[A] { def flatMap[B](f: A => M[B]): M[B] }
def unit[A](x: A): M[A]
*/
fun <A> A.unit(): A? = this
fun <A, B> A?.flatMap(func: (A) -> B?): B? =
if (this != null) func(this) else null
view raw play5.kt hosted with ❤ by GitHub

Looks like Maybe/Option Monad:
assertNull(p1?.flatMap { f -> p2.flatMap { s -> f + s } })
//or
val firstNullable = "param1".unit()?.flatMap { f -> null?.flatMap { f.plus(it) } }assertNull(firstNullable)
//or
val bothAreDefined = "param1".unit()?.flatMap { f -> "param2".unit()?.flatMap { f.plus(it) } }
assertEquals("param1param2", bothAreDefined)
view raw play6.kt hosted with ❤ by GitHub

Looks better - but wouldn't for three and more params. Can we use mapN?
/*
Semigroup's mapN for Pair
*/
fun <A, B> Pair<A?, A?>.mapN(func: (A, A) -> B?): B? =
this.first?.let { a1 -> this.second?.let { a2 -> func(a1, a2) } }
view raw play7.kt hosted with ❤ by GitHub

And the usage is:
val mapNResultWhenNull = Pair(p1, p2).mapN(stringPlus)
assertNull(mapNResultWhenNull)
val mapNResultNotNull = Pair(p2, p3).mapN(stringPlus)
assertNotNull(mapNResultNotNull)
assertEquals("p2p3", mapNResultNotNull)
view raw play8.kt hosted with ❤ by GitHub

Works for Pair type only - for more params we need a custom implementation per type, but idea is clean - Kotlin can support functional solutions with Categories. I'm pretty sure there should  be dozen good Functional Programming libraries in Kotlin like Arrow


Thursday, 4 October 2018

Protobuf getting metadata at runtime

Sometimes it's important to get the list of fields from Protobuf message.

The good example is Kafka with Protobuf serialiser - when the code generator to Java isn't used - but desc file should be generated anyway. It worse to invest into pact testing of the produced messages - check if the number of sent fields is within the range of defined in proto schema.
Unfortunately Protobuf for JVM doesn't support reading metadata from proto files, but if it's compiled to desc file - it's possible to read it via JVM API - that isn't intuitive:

import com.google.protobuf.DescriptorProtos.FileDescriptorSet
import com.google.protobuf.DescriptorProtos.FileDescriptorSet.parseFrom
import java.nio.file.{ Files, Paths }
import scala.collection.JavaConverters._
/**
* Reads protobuf desc file
* @param resourcePath2DescFile the path in resources to protobuf file
*/
class ProtobufReader(resourcePath2DescFile: String) {
private val fileSet: FileDescriptorSet =
parseFrom(Files.readAllBytes(Paths.get(this.getClass.getClassLoader.getResource(resourcePath2DescFile).toURI)))
assert(fileSet.getFileCount == 1, "Protobuf desc file should contain only one desc")
/**
* Returns the list of field names from message definition
* @param messageName message name definded in proto file
*/
def getFields(messageName: String): Seq[String] =
fileSet.getFile(0).getMessageTypeList.asScala
.find(_.getName == messageName)
.map(_.getFieldList.asScala).toList.flatten
.map(_.getName)
}