Improving Scala sorting DSL

Today I encountered an annoying lacking in Scala list sorting support. I just wanted to sort my list by using different fields, some causing an ascending order others causing a descending order.

Well, everything went happily (though with ugly syntax) with Scala´s List.sortBy and unary minus operators... until I needed to perform mixed ascending/descending sorts with strings. Not so easy to implement elegantly.

I tried to google answers but didn't find any useful results. That´s why I decided to make my own syntax by using generics, Ordered trait inheritance and Scala´s implicit Ordering support. I ended up making the following implementation:

	object OrderingDSL {
	case class AscendingOrder[T](value: T)(implicit ord: Ordering[T]) extends Ordered[AscendingOrder[T]] {
	def compare(that: AscendingOrder[T]): Int = ord.compare(value, that.value)
	}
	case class DescendingOrder[T](value: T)(implicit ord: Ordering[T]) extends Ordered[DescendingOrder[T]] {
	def compare(that: DescendingOrder[T]): Int = -ord.compare(value, that.value)
	}
	def asc[T](value: T)(implicit ord: Ordering[T]) = AscendingOrder(value)
	def desc[T](value: T)(implicit ord: Ordering[T]) = DescendingOrder(value)
	}

view raw OrderingDSL.scala hosted with ❤ by GitHub

Now I can sort my collections (at least in Scala 2.10) with neat syntax:
myList.sortBy(v => (asc(v.mem1), desc(v.mem2), ...))

Syntax should work with all classes having Scala Ordering trait implemented (at least all built-in basic types in Scala). Here is a working example how to use the DSL:

	object OrderingDSLExample extends App {
	case class My(num: Int, str: String)
	val myValues = List(My(3, "foo"), My(3, "bar"), My(5, "bar"), My(5, "foo"))
	import OrderingDSL._
	require(myValues.sortBy(m => (asc(m.num), asc(m.str))) ==
	List(My(3, "bar"), My(3, "foo"), My(5, "bar"), My(5, "foo")))
	require(myValues.sortBy(m => (asc(m.num), desc(m.str))) ==
	List(My(3, "foo"), My(3, "bar"), My(5, "foo"), My(5, "bar")))
	require(myValues.sortBy(m => (desc(m.num), asc(m.str))) ==
	List(My(5, "bar"), My(5, "foo"), My(3, "bar"), My(3, "foo")))
	require(myValues.sortBy(m => (desc(m.num), desc(m.str))) ==
	List(My(5, "foo"), My(5, "bar"), My(3, "foo"), My(3, "bar")))
	println("Horray!")
	}

view raw OrderingDSLExample.scala hosted with ❤ by GitHub

I hope this post helps. Happy sorting!

Automatic Backbone model validation

Today I'm going to dig into Backbone.js validations and how to automate them. Backbone itself doesn't provide any proper validation. Fortunately, there is quite good plugin: Backbone.validation. It provides an easy way to define own constraints and error messages for models.

However, Backbone.validation plugin has one disadvantage. It doesn't provide any good way to transfer validation messages across multiple Backbone views nor provide events for single attribute validations. In simple applications this is not a problem, but huge user interface must be divided into multiple views and validation must work automatically for all of those, without generating too much processing overhead.

The solution?

First step was to implement per-attribute. Actually, Backbone.validation provides per-attribute validation callbacks but those are always bound to a view. In order to make our validations more modular, per-attribute validation events must be view-independent. We can create our own events "valid:<attr>" and "invalid:<attr>" by overriding the Backbone.Validation.mixin.validate function:

	# PART 1: tweak backbone validation a little bit
	# ===============================================
	# first, allow errorous inputs to update our model
	Backbone.Validation.configure
	forceUpdate: true
	# then disable default callbacks for backbone validation events
	_.extend Backbone.Validation.callbacks,
	valid: (view, attr, selector) ->
	# these are not used
	invalid: (view, attr, error, selector) ->
	# these are not used
	# we are using our custom model events "valid:<attr>" and "invalid:<attr>" to indicate
	# when model field is validated
	attrValid = (view, attr, selector) ->
	# 'view' is not used, views use our implementation ;)
	@trigger "valid:#{attr}", attr, selector
	attrInvalid = (view, attr, errorMsg, selector) ->
	# 'view' is not used, views use our implementation ;)
	@trigger "invalid:#{attr}", attr, errorMsg, selector
	origValidate = Backbone.Validation.mixin.validate
	Backbone.Validation.mixin.validate = (attrs, setOptions) ->
	options = setOptions || {}
	# "this" refers now to model so we apply it to our validation
	# functions, thus can trigger validation events for right model
	options.valid = => attrValid.apply(@, arguments)
	options.invalid = => attrInvalid.apply(@, arguments)
	origValidate.call(@, attrs, options)
	# add customized validation to our models, when validations occur, they raise
	# model events valid:<attr> and invalid:<attr> to our models. Views may be interested
	# in those validation callbacks for showing error messages.
	_.extend(Backbone.Model.prototype, Backbone.Validation.mixin)

view raw validation-improvements.coffee hosted with ❤ by GitHub

Ok. Now we have view independent validation events for attributes. Next step is to bind those events into views so that each input listens valid and invalid events from associated models. The problem is that there is no pre-defined information about available events because they are generated dynamically from model constraints. Thus, I've looped every input element from the view after render and used their name attribute for event binding:

	# PART 2: bind our backbone view to validation events and also for
	# input changes so that model changes automatically when input changes
	# =======================================================================
	# This defines which element's attribute will be used for linking
	inputSelector = 'name'
	# model update function, which is called when input changes
	updateModel = (model, attr, value) ->
	model.set(attr, value || null)
	# This is called when input element is marked as valid
	# It basically just removes Bootstrap error classes and
	# custom error messages from input's control-group
	setValid = ($input, attr, selector) ->
	$group = $input.closest('.control-group')
	return if $group.length == 0
	$group.find('.help-inline.error').remove()
	$group.removeClass('error')
	return
	# Helper function to check if input has already same an error message
	containsError = ($err, $group) ->
	errText = $err.text()
	for help in $group.find('.help-inline.error')
	return true if $(help).text() == errText
	return false
	# This is called when input is marked as valid
	setInvalid = ($input, attr, errMsg, selector) ->
	$group = $input.closest('.control-group')
	if $group.length == 0
	console.log "Invalid input '#{attr}' has no control group. No message will be shown..", $input
	return
	$controls = $group.find('.controls')
	$err = $("<span class='help-inline error'>#{_.escape(errMsg)}</span>")
	if !containsError($err, $group)
	if $controls.length > 0
	$err.insertAfter($controls).addClass('next-line')
	else
	$group.append($err)
	# finally add error class
	$group.addClass('error')
	# This function is called when new stuff is rendered to the document
	# It searches all inputs and binds update and validation events to
	# those inputs so that changes/validation results are reflected automatically
	# between model <-> view <-> template
	addInputBindings = (view, model) ->
	return if not model
	view.$(':input:not(.btn)').each ->
	$this = $(this)
	return if not $this.attr(inputSelector)
	# if we have backbone relational / nested plugin then we may have want
	# to validate and update also nested attributes
	attr = $this.attr(inputSelector).replace(/\-/g, '.')
	# input change to model bindings
	if !$this.data('ignore-model')
	# when input changes, then models changes too
	# we could add also events from model changes to change
	# input value but we won't do it in this example
	$this.on 'change', -> updateModel(model, attr, $this.val())
	# validation bindings
	if !$this.data('ignore-validation')
	# remove previous
	model.off "valid:#{attr}", null, view
	model.off "invalid:#{attr}", null, view
	# add new
	model.on "valid:#{attr}", ((attr, selector) -> setValid($this, attr, selector)), view
	model.on "invalid:#{attr}", ((attr, errMsg, selector) -> setInvalid($this, attr, errMsg, selector)), view
	# This is base class for all our items
	class window.ItemView extends Backbone.Marionette.ItemView
	constructor: (options) ->
	super
	@on 'render', => addInputBindings(@, @model)

view raw view-improvements.coffee hosted with ❤ by GitHub

In the above code, I've added also an automatic input-to-model binding so that every time when user changes the input, the changed value will be stored into associated model attribute. Validation binding and input-to-model bindings can be disabled from individual inputs by defining HTML5 data-attributes ignore-model="true" and ignore-validation="true".

Because these kind of validation and input-to-model bindings are kind of boilerplate code, I wanted to automate them by using CoffeeScript inheritance and Backbone.marionette plugin: all that developers must do is to inherit their new view classes from ItemView class and after that all validations and input-to-model synchronizations are working automagically! Cool, I think!

And usage...

Here is a little example code about the usage of our brand new "framework":

	jQuery ->
	# Simple person model with two attributes
	class Person extends Backbone.Model
	validation:
	age:
	required: true
	min: 10
	name:
	required: true
	msg: "All persons have a name..."
	class PersonView extends ItemView
	el: $("#person-form")
	template: Handlebars.compile($("#person-form-template").html())
	events:
	"click #send-btn" : 'send'
	"click #cancel-btn" : 'cancel'
	send: ->
	@model.validate()
	if @model.isValid()
	alert "OK!"
	# do some save stuff here
	cancel: ->
	@model.clear()
	@render()
	view = new PersonView(model: new Person())
	view.render()

view raw example-usage.coffee hosted with ❤ by GitHub

Of course our framework requires that our markup follows some specific conventions. I'm using Twitter Bootstrap so the framework expects that every validated input is inside control-group div. However, it shouldn't be non-trivial to tweak setValid and setInvalid methods form part 2 in order to make your own markup implementation. ;)

A working example can be found from jsFiddle: http://jsfiddle.net/RfcMK/4/

Don't shit your Grails URL reloading with extreme paths!

Gosh! I hope that the topic will explain itself properly. Do not ever put your Grails project behind a path that contains other characters than [a-zA-Z0-9_.]!

Yesterday I spent several hours trying to figure out why my controller/action/UrlMapping changes are not reflecting into my run-time paths. For example, I changed my FooController's action from "bar" to "foo". The corresponding URL should have changed from <app>/foo/bar to <app>/foo/foo. Well.. it didn't.. However, my Grails console told me that changes have been compiled. If I changed the actual logic inside "bar" action, the changes were applied immediately. Strange indeed!

Power Googling didn't give any results. Neither did Grails version changes (I tried 2.2.1, 2.1.4 and 2.0.4). I suspected that the cause was my Java version (1.7.0_17), because my other IDE with Java 1.6.0_24 was working fine. It wasn't.

Finally I figured that my Eclipse workspace was set to C:/.../Some(thing)/../workspace. So, I moved my workspace to a different path and tried again. Now it worked like a charm! So the lesson of this blog post is: never shit your Grails URL reloading with extreme paths!

Tuning Grails resources plugin to support multiple modules for same file

A long time since my previous blog post. Deepest apologizes, I have had so little time for my "hobbies"...

But now! I faced so frustrating issue in my work so that I must forward it to you. We have been developing our system quite a time already. Real problems started when it was time for the first test deployment to the actual server. We are using Grails as our controller layer which includes a wonderful resource plugin that enables us to use newest techiques in our assets easily (such as Coffeescript or LESS). The production mode will be caching, zipping and minimizing all resource modules in order to reduce clients' overhead. These are not used in development mode. So, during the deployment I discovered that the resource plugin has a "bug" which prevents the definition of same file to multiple modules.

After a Google moment, I found that the plugin's author wasn't planning to support such a feature (GPRESOURCES-166). We had two options: either divide our code into multiple mini-mini modules (which reduces the usefulness of modules a lot) OR try to go around the bug/feature.

So, after a long moment of discovering plugin's source codes, I managed to find a detour. It is not an elegant way but working well enough. The resouces plugin uses a chain of resource mappers to transform original files into processed files. The bundling and bundle renamining (for cache prevention) are dealt by using file paths as identifiers for files. So so... The best bet is to rename the file with a unique prefix before the resource is "registered" into module. When registering is done, the resources plugin tries to fetch the resource file based on its defined filename. Thus, we need to translate the modified filename back to the original one so that resources plugin can find the actual file. When the file has been fetched, resources plugin starts to process the file with its processing chain. We must add our own custom resource mapper to the beginning of that chain to translate processed resource's URL back to the original so that later processors can deal with right files.

The solution is to override two resource plugin's methods by using meta-programming. First we modify the filename during the resource definition. Then we rename it back to the original for file handle fetching. After that, we use our own resource mapper to change resource's URLs back to the normal at the beginning of the processing chain. And hóle! Now the plugin supports multiple module definitions for same file also with cached-resources plugin.

Here is the actual code:

	package com.tunkkaus
	import org.grails.plugin.resource.ResourceMeta
	import org.grails.plugin.resource.ResourceProcessor
	import org.grails.plugin.resource.mapper.MapperPhase;
	import org.grails.plugin.resource.module.ModuleBuilder
	import org.springframework.util.AntPathMatcher
	/**
	* We are using prefix '_' because we want that this resource mapper
	* is FIRST mapper of the processing chain. Sorting is done by name
	* so '_' prefix gives us advantage compared to other mappers.
	*
	*/
	class _MultiModuleResourceMapper {
	private static final String MULTIPLE_RESOURCES_PREFIX = "/__multiModule__"
	private static long declaredResourcesNum = 0
	static defaultIncludes = [ '**/*.coffee', '**/*.js', '**/*.handlebars' ]
	static String getResourcePrefix() { "${MULTIPLE_RESOURCES_PREFIX}.${declaredResourcesNum++}/" }
	static boolean isMultipleResourceFile(String uri) { uri?.startsWith(MULTIPLE_RESOURCES_PREFIX) }
	static String getOriginalURI(String uri) { uri?.replaceFirst(MULTIPLE_RESOURCES_PREFIX + '\\.\\d+/', '/') }
	private static boolean canBeRenamed(String filepath) {
	if(filepath.startsWith(MULTIPLE_RESOURCES_PREFIX)) return false
	// behaviour can be modified by chancing defaultIncludes
	def matcher = new AntPathMatcher()
	return defaultIncludes.find { matcher.match(it, filepath) } != null
	}
	/**
	* Call this from resources.groovy! This modifies resources plugin methods
	* so that they support multiple resource definitions inside multiple modules.
	*/
	static void enable() {
	def origResource = ModuleBuilder.metaClass.getMetaMethod("resource", [Object] as Object[])
	def origGetURL = ResourceProcessor.metaClass.getMetaMethod("getOriginalResourceURLForURI", [Object] as Object[])
	assert origResource != null && origGetURL != null
	// override modulebuilder's resource function to apply a unique prefix
	// because file will have an unique prefix, then it is interpreted as
	// "new" file for resources plugin
	ModuleBuilder.metaClass.resource = { attrs ->
	if (attrs instanceof String) {
	if (canBeRenamed(attrs)) {
	attrs = resourcePrefix + attrs
	}
	} else if (attrs instanceof Map && attrs.url instanceof String) {
	if (canBeRenamed(attrs.url)) {
	attrs.url = resourcePrefix + attrs.url
	}
	}
	origResource.invoke(delegate, attrs)
	}
	// this method is called before resource processing chain begins
	// this gets the file handle to the original resource. Since we have
	// added a prefix, the actual file is never found unless we tweak this
	// method a little bit to transform the modified filename back to the
	// original one (just for this method)
	ResourceProcessor.metaClass.getOriginalResourceURLForURI { uri ->
	if (isMultipleResourceFile(uri)) {
	uri = getOriginalURI(uri)
	}
	return origGetURL.invoke(delegate, uri)
	}
	}
	// using first possible mapper phase
	def phase = MapperPhase.GENERATION
	def map(resource, config) {
	def url = resource?.actualUrl
	// now we can process the resource whatever we want. Let's just change
	// resource's URLs back to the original file's, so that later processors
	// can deal with right file.
	if (isMultipleResourceFile(url)) {
	url = getOriginalURI(url)
	resource.actualUrl = resource.originalUrl = resource.sourceUrl = url
	}
	}
	}

view raw _MultiModuleResourceMapper.groovy hosted with ❤ by GitHub

You must place that into your grails-app/resourceMappers folder. After that you must enable support by calling MultipleStaticResources.enable() from your resources.groovy bean definition closure.

	import com.tunkkaus._MultiModuleResourceMapper
	beans = {
	_MultiModuleResourceMapper.enable()
	}

view raw resources.groovy hosted with ❤ by GitHub

Of course this solution may conflict with our own resource mappers, but for me it works well enough: resource mappers for CoffeeScript, Handlebars, LESS, zipped and bundled/cached resoures (and also few own) are working fine with this hack.

Happy hacking!

CoffeeScript aspects

Hojoo! First "real" blog message coming, finally!

There are some cases when client implementation shoudn't be available for everyone. I encountered such a problem when I was creating Backbone views for different users: users may have different roles, and depending on those roles, some actions should be available or not.

Of course I could have just ignored the actions is my Javascript. In this case, the whole implementation had to be hidden so if-else was out of question. Making different views for different role combinations is just plain stupid.

I decided to solve the problem by using using aspect oriented programming. Javascript functions have apply-method so it was quite trivial to make a simple aspect library:

	class MyClass
	foo: ->
	alert "foobar"
	bar: ->
	alert "barfoo"
	@foo()
	aspect = (clz, extensions) ->
	weaver =
	# this does the actual binding
	# this is called from extension code
	around: (pointcut, advice) ->
	prev = null
	prev = if clz.prototype then clz.prototype[pointcut] else null
	return if not prev
	clz.prototype[pointcut] = ->
	args = arguments
	obj = this
	joinPoint =
	getTarget: -> obj
	retVal: null
	getArgs: -> args
	getReturnValue: -> @retVal
	setReturnValue: (val) -> @retVal = val
	# this is called from advice (usually) once
	proceed: ->
	@setReturnValue prev.apply(obj, @getArgs())
	# this line calls the advice code when
	# target object method is called
	advice.call(weaver, joinPoint)
	return joinPoint.getReturnValue()
	# bind advices
	extensions.apply(weaver)
	aspect MyClass, ->
	@around 'foo', (jp) ->
	alert "BEFORE"
	jp.proceed()
	alert "AFTER"
	a = new MyClass
	a.foo()
	alert "---"
	a.bar()

view raw aspect_example.coffee hosted with ❤ by GitHub

Usage is trivial: just call @around-method and give the function name (pointcut) which must be wrapped. When function is called, the wrapper code is called first. It can decide whether to execute the original code via joinpoint interface or not. It can also modify function call argumens and return value. Joinpoint supports following functions:

	class MyClass
	simple: ->
	alert "hello!"
	another: ->
	alert "hi!"
	withParams: (a, b) ->
	alert a + " - " + b
	withReturnValue: (a) ->
	return 2 * a
	...
	aspect MyClass, ->
	@around 'simple', (jp) ->
	alert "!!"
	jp.proceed()
	a = new MyClass
	a.simple()
	....
	# multiple aspects are also ok! these aspects could be in separate files
	# not that the call order depends on execution order: latest advice is called first
	aspect MyClass, ->
	@around 'simple', (jp) ->
	alert "!!"
	jp.proceed()
	aspect MyClass, ->
	@around 'simple', (jp) ->
	jp.proceed()
	alert '??'
	a = new MyClass
	a.simple()
	...
	# we can add many advices in same aspect
	aspect MyClass, ->
	@around 'simple', (jp) -> alert '!!'; jp.proceed()
	@around 'another', (jp) ->
	# we can also "override" whole implementation
	alert "overrided"
	a = new MyClass
	a.simple()
	a.another()
	...
	# we can modify arguments
	aspect MyClass, ->
	@around 'withParams', (jp) ->
	for i in [0..1]
	jp.getArgs()[i] = "*#{jp.getArgs()[i]}*"
	jp.proceed()
	a = new MyClass
	a.withParams("foo", "bar")
	...
	# we can also modify return values
	aspect MyClass, ->
	@around 'withReturnValue', (jp) ->
	jp.proceed()
	jp.setReturnValue Math.abs(jp.getReturnValue())
	a = new MyClass
	alert a.withReturnValue(3)
	alert a.withReturnValue(-3)

view raw gistfile1.coffee hosted with ❤ by GitHub

Now I just have to make a basic Backbone view which is public for everyone. When I fetch the roles during page render, I can include all needed logic in separate files. Those files add the logic by binding their advices to the basic view. I can also apply some security policy for those Javascript "extensions", completely hiding the "secret features" from unwanted users. Pretty handy, I think! :)

Some definitions

Hi,

It's time to clarify the story behind the name of this blog. As you can guess, tunkkaus is a Finnish word. The direct translation of 'tunkkaus' is that someone is using a jack (tunkki = jack) to lift a car before changing its tyre.

However, when talking with other software engineering professionals, the word 'tunkkaus' may have an another meaning. When I say that I'm going to 'tunkata' the software somehow, it means that I'm going to develop the software, usually with some exotic solutions. These solutions may be marvellous or spaghetti or something just plain normal. Usually the word has a little bit negative tone when somebody says it. However, it can have also a positive tone.

Thus, tunkkaus is the way to go!

Image: http://forum.f-bodyfinland.com/Yabbse/index.php?topic=59966.15

Greetings!

Hello folks!

Here is some background about me: I'm a software engineer and a computer science student from Aalto University Finland.

Coding is my passion - every commit I make I want to be the perfect one. I love beautiful code and genious solutions. I started my programming "career" when I was 12 years old. My first language was a BASIC language, CoolBasic. After that I moved on to C++ world and making games for my fun. By that time, C++ was the best language ever... at least in my opinion. That was until I discovered C# (I think it was .NET 3.0) which leaded me into carbage collected world. Java was the next one to learn. I wasn't so interested about Java when I begun my school at Aalto, but after a while I started to like it too...

I've always considered myself as a backend developer. That was what I said to my boss when I started my work in ICT cosulting company about two years ago. However, work challenges leaded me into Grails and dynamic programming. WOW! I must say that Grails/Groovy was the thing. Now I can't even think about making software without funcional languages. I've found frontend more and more interesting. Thanks to my collegue, I discovered Coffeescript which also is awesome! I must say that my "backend dream" is inevitably slipping towards front. Thus, I'll most likely cover some CoffeeScript/Grails/Groovy stuff in this blog.

I've noticed in my work that you can do things in two ways: right and wrong. Thus, I try to make this world a little bit better by telling you some of my experiences from the software engineering field. I hope that my experiences and adversities will motive you readers to select the right choices and also make this world better by producing higher quality softwares.

Cheers,
Matti