Improving Scala sorting DSL

Today I encountered an annoying lacking in Scala list sorting support. I just wanted to sort my list by using different fields, some causing an ascending order others causing a descending order.

Well, everything went happily (though with ugly syntax) with Scala´s List.sortBy and unary minus operators... until I needed to perform mixed ascending/descending sorts with strings. Not so easy to implement elegantly.

I tried to google answers but didn't find any useful results. That´s why I decided to make my own syntax by using generics, Ordered trait inheritance and Scala´s implicit Ordering support. I ended up making the following implementation:

	object OrderingDSL {
	case class AscendingOrder[T](value: T)(implicit ord: Ordering[T]) extends Ordered[AscendingOrder[T]] {
	def compare(that: AscendingOrder[T]): Int = ord.compare(value, that.value)
	}
	case class DescendingOrder[T](value: T)(implicit ord: Ordering[T]) extends Ordered[DescendingOrder[T]] {
	def compare(that: DescendingOrder[T]): Int = -ord.compare(value, that.value)
	}
	def asc[T](value: T)(implicit ord: Ordering[T]) = AscendingOrder(value)
	def desc[T](value: T)(implicit ord: Ordering[T]) = DescendingOrder(value)
	}

view raw OrderingDSL.scala hosted with ❤ by GitHub

Now I can sort my collections (at least in Scala 2.10) with neat syntax:
myList.sortBy(v => (asc(v.mem1), desc(v.mem2), ...))

Syntax should work with all classes having Scala Ordering trait implemented (at least all built-in basic types in Scala). Here is a working example how to use the DSL:

	object OrderingDSLExample extends App {
	case class My(num: Int, str: String)
	val myValues = List(My(3, "foo"), My(3, "bar"), My(5, "bar"), My(5, "foo"))
	import OrderingDSL._
	require(myValues.sortBy(m => (asc(m.num), asc(m.str))) ==
	List(My(3, "bar"), My(3, "foo"), My(5, "bar"), My(5, "foo")))
	require(myValues.sortBy(m => (asc(m.num), desc(m.str))) ==
	List(My(3, "foo"), My(3, "bar"), My(5, "foo"), My(5, "bar")))
	require(myValues.sortBy(m => (desc(m.num), asc(m.str))) ==
	List(My(5, "bar"), My(5, "foo"), My(3, "bar"), My(3, "foo")))
	require(myValues.sortBy(m => (desc(m.num), desc(m.str))) ==
	List(My(5, "foo"), My(5, "bar"), My(3, "foo"), My(3, "bar")))
	println("Horray!")
	}

view raw OrderingDSLExample.scala hosted with ❤ by GitHub

I hope this post helps. Happy sorting!

Automatic Backbone model validation

Today I'm going to dig into Backbone.js validations and how to automate them. Backbone itself doesn't provide any proper validation. Fortunately, there is quite good plugin: Backbone.validation. It provides an easy way to define own constraints and error messages for models.

However, Backbone.validation plugin has one disadvantage. It doesn't provide any good way to transfer validation messages across multiple Backbone views nor provide events for single attribute validations. In simple applications this is not a problem, but huge user interface must be divided into multiple views and validation must work automatically for all of those, without generating too much processing overhead.

The solution?

First step was to implement per-attribute. Actually, Backbone.validation provides per-attribute validation callbacks but those are always bound to a view. In order to make our validations more modular, per-attribute validation events must be view-independent. We can create our own events "valid:<attr>" and "invalid:<attr>" by overriding the Backbone.Validation.mixin.validate function:

	# PART 1: tweak backbone validation a little bit
	# ===============================================
	# first, allow errorous inputs to update our model
	Backbone.Validation.configure
	forceUpdate: true
	# then disable default callbacks for backbone validation events
	_.extend Backbone.Validation.callbacks,
	valid: (view, attr, selector) ->
	# these are not used
	invalid: (view, attr, error, selector) ->
	# these are not used
	# we are using our custom model events "valid:<attr>" and "invalid:<attr>" to indicate
	# when model field is validated
	attrValid = (view, attr, selector) ->
	# 'view' is not used, views use our implementation ;)
	@trigger "valid:#{attr}", attr, selector
	attrInvalid = (view, attr, errorMsg, selector) ->
	# 'view' is not used, views use our implementation ;)
	@trigger "invalid:#{attr}", attr, errorMsg, selector
	origValidate = Backbone.Validation.mixin.validate
	Backbone.Validation.mixin.validate = (attrs, setOptions) ->
	options = setOptions || {}
	# "this" refers now to model so we apply it to our validation
	# functions, thus can trigger validation events for right model
	options.valid = => attrValid.apply(@, arguments)
	options.invalid = => attrInvalid.apply(@, arguments)
	origValidate.call(@, attrs, options)
	# add customized validation to our models, when validations occur, they raise
	# model events valid:<attr> and invalid:<attr> to our models. Views may be interested
	# in those validation callbacks for showing error messages.
	_.extend(Backbone.Model.prototype, Backbone.Validation.mixin)

view raw validation-improvements.coffee hosted with ❤ by GitHub

Ok. Now we have view independent validation events for attributes. Next step is to bind those events into views so that each input listens valid and invalid events from associated models. The problem is that there is no pre-defined information about available events because they are generated dynamically from model constraints. Thus, I've looped every input element from the view after render and used their name attribute for event binding:

	# PART 2: bind our backbone view to validation events and also for
	# input changes so that model changes automatically when input changes
	# =======================================================================
	# This defines which element's attribute will be used for linking
	inputSelector = 'name'
	# model update function, which is called when input changes
	updateModel = (model, attr, value) ->
	model.set(attr, value || null)
	# This is called when input element is marked as valid
	# It basically just removes Bootstrap error classes and
	# custom error messages from input's control-group
	setValid = ($input, attr, selector) ->
	$group = $input.closest('.control-group')
	return if $group.length == 0
	$group.find('.help-inline.error').remove()
	$group.removeClass('error')
	return
	# Helper function to check if input has already same an error message
	containsError = ($err, $group) ->
	errText = $err.text()
	for help in $group.find('.help-inline.error')
	return true if $(help).text() == errText
	return false
	# This is called when input is marked as valid
	setInvalid = ($input, attr, errMsg, selector) ->
	$group = $input.closest('.control-group')
	if $group.length == 0
	console.log "Invalid input '#{attr}' has no control group. No message will be shown..", $input
	return
	$controls = $group.find('.controls')
	$err = $("<span class='help-inline error'>#{_.escape(errMsg)}</span>")
	if !containsError($err, $group)
	if $controls.length > 0
	$err.insertAfter($controls).addClass('next-line')
	else
	$group.append($err)
	# finally add error class
	$group.addClass('error')
	# This function is called when new stuff is rendered to the document
	# It searches all inputs and binds update and validation events to
	# those inputs so that changes/validation results are reflected automatically
	# between model <-> view <-> template
	addInputBindings = (view, model) ->
	return if not model
	view.$(':input:not(.btn)').each ->
	$this = $(this)
	return if not $this.attr(inputSelector)
	# if we have backbone relational / nested plugin then we may have want
	# to validate and update also nested attributes
	attr = $this.attr(inputSelector).replace(/\-/g, '.')
	# input change to model bindings
	if !$this.data('ignore-model')
	# when input changes, then models changes too
	# we could add also events from model changes to change
	# input value but we won't do it in this example
	$this.on 'change', -> updateModel(model, attr, $this.val())
	# validation bindings
	if !$this.data('ignore-validation')
	# remove previous
	model.off "valid:#{attr}", null, view
	model.off "invalid:#{attr}", null, view
	# add new
	model.on "valid:#{attr}", ((attr, selector) -> setValid($this, attr, selector)), view
	model.on "invalid:#{attr}", ((attr, errMsg, selector) -> setInvalid($this, attr, errMsg, selector)), view
	# This is base class for all our items
	class window.ItemView extends Backbone.Marionette.ItemView
	constructor: (options) ->
	super
	@on 'render', => addInputBindings(@, @model)

view raw view-improvements.coffee hosted with ❤ by GitHub

In the above code, I've added also an automatic input-to-model binding so that every time when user changes the input, the changed value will be stored into associated model attribute. Validation binding and input-to-model bindings can be disabled from individual inputs by defining HTML5 data-attributes ignore-model="true" and ignore-validation="true".

Because these kind of validation and input-to-model bindings are kind of boilerplate code, I wanted to automate them by using CoffeeScript inheritance and Backbone.marionette plugin: all that developers must do is to inherit their new view classes from ItemView class and after that all validations and input-to-model synchronizations are working automagically! Cool, I think!

And usage...

Here is a little example code about the usage of our brand new "framework":

	jQuery ->
	# Simple person model with two attributes
	class Person extends Backbone.Model
	validation:
	age:
	required: true
	min: 10
	name:
	required: true
	msg: "All persons have a name..."
	class PersonView extends ItemView
	el: $("#person-form")
	template: Handlebars.compile($("#person-form-template").html())
	events:
	"click #send-btn" : 'send'
	"click #cancel-btn" : 'cancel'
	send: ->
	@model.validate()
	if @model.isValid()
	alert "OK!"
	# do some save stuff here
	cancel: ->
	@model.clear()
	@render()
	view = new PersonView(model: new Person())
	view.render()

view raw example-usage.coffee hosted with ❤ by GitHub

Of course our framework requires that our markup follows some specific conventions. I'm using Twitter Bootstrap so the framework expects that every validated input is inside control-group div. However, it shouldn't be non-trivial to tweak setValid and setInvalid methods form part 2 in order to make your own markup implementation. ;)

A working example can be found from jsFiddle: http://jsfiddle.net/RfcMK/4/

Don't shit your Grails URL reloading with extreme paths!

Gosh! I hope that the topic will explain itself properly. Do not ever put your Grails project behind a path that contains other characters than [a-zA-Z0-9_.]!

Yesterday I spent several hours trying to figure out why my controller/action/UrlMapping changes are not reflecting into my run-time paths. For example, I changed my FooController's action from "bar" to "foo". The corresponding URL should have changed from <app>/foo/bar to <app>/foo/foo. Well.. it didn't.. However, my Grails console told me that changes have been compiled. If I changed the actual logic inside "bar" action, the changes were applied immediately. Strange indeed!

Power Googling didn't give any results. Neither did Grails version changes (I tried 2.2.1, 2.1.4 and 2.0.4). I suspected that the cause was my Java version (1.7.0_17), because my other IDE with Java 1.6.0_24 was working fine. It wasn't.

Finally I figured that my Eclipse workspace was set to C:/.../Some(thing)/../workspace. So, I moved my workspace to a different path and tried again. Now it worked like a charm! So the lesson of this blog post is: never shit your Grails URL reloading with extreme paths!

Tuning Grails resources plugin to support multiple modules for same file

A long time since my previous blog post. Deepest apologizes, I have had so little time for my "hobbies"...

But now! I faced so frustrating issue in my work so that I must forward it to you. We have been developing our system quite a time already. Real problems started when it was time for the first test deployment to the actual server. We are using Grails as our controller layer which includes a wonderful resource plugin that enables us to use newest techiques in our assets easily (such as Coffeescript or LESS). The production mode will be caching, zipping and minimizing all resource modules in order to reduce clients' overhead. These are not used in development mode. So, during the deployment I discovered that the resource plugin has a "bug" which prevents the definition of same file to multiple modules.

After a Google moment, I found that the plugin's author wasn't planning to support such a feature (GPRESOURCES-166). We had two options: either divide our code into multiple mini-mini modules (which reduces the usefulness of modules a lot) OR try to go around the bug/feature.

So, after a long moment of discovering plugin's source codes, I managed to find a detour. It is not an elegant way but working well enough. The resouces plugin uses a chain of resource mappers to transform original files into processed files. The bundling and bundle renamining (for cache prevention) are dealt by using file paths as identifiers for files. So so... The best bet is to rename the file with a unique prefix before the resource is "registered" into module. When registering is done, the resources plugin tries to fetch the resource file based on its defined filename. Thus, we need to translate the modified filename back to the original one so that resources plugin can find the actual file. When the file has been fetched, resources plugin starts to process the file with its processing chain. We must add our own custom resource mapper to the beginning of that chain to translate processed resource's URL back to the original so that later processors can deal with right files.

The solution is to override two resource plugin's methods by using meta-programming. First we modify the filename during the resource definition. Then we rename it back to the original for file handle fetching. After that, we use our own resource mapper to change resource's URLs back to the normal at the beginning of the processing chain. And hóle! Now the plugin supports multiple module definitions for same file also with cached-resources plugin.

Here is the actual code:

	package com.tunkkaus
	import org.grails.plugin.resource.ResourceMeta
	import org.grails.plugin.resource.ResourceProcessor
	import org.grails.plugin.resource.mapper.MapperPhase;
	import org.grails.plugin.resource.module.ModuleBuilder
	import org.springframework.util.AntPathMatcher
	/**
	* We are using prefix '_' because we want that this resource mapper
	* is FIRST mapper of the processing chain. Sorting is done by name
	* so '_' prefix gives us advantage compared to other mappers.
	*
	*/
	class _MultiModuleResourceMapper {
	private static final String MULTIPLE_RESOURCES_PREFIX = "/__multiModule__"
	private static long declaredResourcesNum = 0
	static defaultIncludes = [ '**/*.coffee', '**/*.js', '**/*.handlebars' ]
	static String getResourcePrefix() { "${MULTIPLE_RESOURCES_PREFIX}.${declaredResourcesNum++}/" }
	static boolean isMultipleResourceFile(String uri) { uri?.startsWith(MULTIPLE_RESOURCES_PREFIX) }
	static String getOriginalURI(String uri) { uri?.replaceFirst(MULTIPLE_RESOURCES_PREFIX + '\\.\\d+/', '/') }
	private static boolean canBeRenamed(String filepath) {
	if(filepath.startsWith(MULTIPLE_RESOURCES_PREFIX)) return false
	// behaviour can be modified by chancing defaultIncludes
	def matcher = new AntPathMatcher()
	return defaultIncludes.find { matcher.match(it, filepath) } != null
	}
	/**
	* Call this from resources.groovy! This modifies resources plugin methods
	* so that they support multiple resource definitions inside multiple modules.
	*/
	static void enable() {
	def origResource = ModuleBuilder.metaClass.getMetaMethod("resource", [Object] as Object[])
	def origGetURL = ResourceProcessor.metaClass.getMetaMethod("getOriginalResourceURLForURI", [Object] as Object[])
	assert origResource != null && origGetURL != null
	// override modulebuilder's resource function to apply a unique prefix
	// because file will have an unique prefix, then it is interpreted as
	// "new" file for resources plugin
	ModuleBuilder.metaClass.resource = { attrs ->
	if (attrs instanceof String) {
	if (canBeRenamed(attrs)) {
	attrs = resourcePrefix + attrs
	}
	} else if (attrs instanceof Map && attrs.url instanceof String) {
	if (canBeRenamed(attrs.url)) {
	attrs.url = resourcePrefix + attrs.url
	}
	}
	origResource.invoke(delegate, attrs)
	}
	// this method is called before resource processing chain begins
	// this gets the file handle to the original resource. Since we have
	// added a prefix, the actual file is never found unless we tweak this
	// method a little bit to transform the modified filename back to the
	// original one (just for this method)
	ResourceProcessor.metaClass.getOriginalResourceURLForURI { uri ->
	if (isMultipleResourceFile(uri)) {
	uri = getOriginalURI(uri)
	}
	return origGetURL.invoke(delegate, uri)
	}
	}
	// using first possible mapper phase
	def phase = MapperPhase.GENERATION
	def map(resource, config) {
	def url = resource?.actualUrl
	// now we can process the resource whatever we want. Let's just change
	// resource's URLs back to the original file's, so that later processors
	// can deal with right file.
	if (isMultipleResourceFile(url)) {
	url = getOriginalURI(url)
	resource.actualUrl = resource.originalUrl = resource.sourceUrl = url
	}
	}
	}

view raw _MultiModuleResourceMapper.groovy hosted with ❤ by GitHub

You must place that into your grails-app/resourceMappers folder. After that you must enable support by calling MultipleStaticResources.enable() from your resources.groovy bean definition closure.

	import com.tunkkaus._MultiModuleResourceMapper
	beans = {
	_MultiModuleResourceMapper.enable()
	}

view raw resources.groovy hosted with ❤ by GitHub

Of course this solution may conflict with our own resource mappers, but for me it works well enough: resource mappers for CoffeeScript, Handlebars, LESS, zipped and bundled/cached resoures (and also few own) are working fine with this hack.

Happy hacking!