Which for some reason is not in the abnf spec, and thus will be ignored up until somebody tries to implement this and fails a compatibility test.
Please put it in your abnf spec. People use them, you know.
Yes, it's hard to specify. The problem is that you have both an uncapped ID and an uncapped PAYLOAD in the same message. I recommend giving ID a max length of, say, 32, and PAYLOAD then has a max length of 951 if I'm counting right.
Or you could consider that an IPv6 path MTU is at least 1280, and use that (or 1232) as your per message bound instead of 1024. You're sending a packet, might as well get full value.
> The comma makes it slightly simpler for humans to read netstrings that are used as adjacent records, and provides weak verification of correct parsing.
A big advantage of LF-delimited over length-prefixed messages is netcat/telnet-friendliness. That was more valuable to us than being binary-clean as our use cases do not involve sending large binary messages.
I think you might want to make a distinction between a stream packet and a completed message.
If you're going for telnet compatibility then you'll want to terminate packets in CR+LF, but possibly expect to see only CR or LF from the client (ASCII mode).
Your stream could either be stateful (a message is always sent complete and in order, even if it takes multiple stream packets) or stateless* (different messages might have stream packets consecutively).
It would be more future proof if you started with a message grammar and then defined your protocol on top of that.
- golang-builder does not provide a convenient script, instead requiring manual
docker invocations [more flexible but more cumbersome]
- golang-builder uses volumes, which makes it very hard to use within a container
[may not matter to everyone but it certainly does to us]
- golang-builder uses a significantly larger base image
- gockerize has a really nice logo ;)
> This isn't so much Java vs. Go as it is JIT/interpreted vs. AOT-compiled. The numbers are entirely typical across a wide range of such comparisons.
> [...] while an app written in C++, Rust, or Go will take 2MB.
Agreed. As mentioned in the blog post, I considered Rust but decided against it because of I found it much less mature than Go. I did not consider C++ because, as mentioned in the blog post, part of the point was to experiment with new language/tools and even though I consider myself proficient with it, I learned the hard way that the lack of memory safety is rarely worth it.
> I suspect that the 668MB Java image was at least 90% unnecessary garbage that was not actually needed at runtime. Unfortunately the package managers we all use are not optimized for containers
Exactly. That was part of the point of this blog post, which I may not have been successful at getting across. Switching to go was, if not the path of least resistance to solve this issue, at least one of a few relatively easy routes. It also happened to be a great deal of fun.
I assume you didn't use the normal Dockerfile-based build system to build your super-small Docker images for the Go-based services. So how did you do it?
We're not doing anything too fancy. Basically, we spawn a container to build a statically linked binary and do a regular Dockerfile-based build inside that container. The result is an image which contains only a single binary (any maybe some static assets like config files or images).
We're planning to open source our build script shortly.
Hehe, yes, I'm afraid I have a proclivity for terrible puns. I do think they're somewhat less cringe-worthy that the link-bait titles that are all the rage these days (or at least endearingly cringeworthy).
The footprint of this particular service could have been optimized in Java but:
1. the JVM itself imposes a high floor (hotspot, many shared libs loaded, ...)
2. the Java language is full of overhead at every level (boxed types are a pet peeve of mine)
3. the Java ecosystem has a tendency to regard memory as an inexhaustible resource, which lead a lot of waste in many 3rd party libraries
The core point is that optimizing this particular Java program (and the others that followed) would have been more time-consuming than a Golang rewrite and would have probably increased the complexity whereas a Golang rewrite reduced it.
Optimization was the original goal, increased maintainability was a pleasant result.
Point 2... yeah, that's one of my major problems with Java. Unnecessary boxing and the unreasonable amount of complexity if you work around it. In any type of Object collections, it consumes memory, stresses garbage collector unnecessarily and causes a lot of CPU cache misses.
Value types would help so much with this issue. I know they're coming one day. I hope Java/JVM can replicate memory efficiency and cache coherence of C++ std::vector for small objects.
Buffer overflow concerns are not applicable to SSMP however, as the spec explicitly restricts message size to a maximum of 1024 bytes.