Today's graphics breakthrough!

One of the bits of software that I play around with as a background task is Apple's truly astonishing 'Quartz Composer'. Dismissed as a toy by many people, it repays careful study many times over.

My main area of research using Quartz Composer is music visualisation: turning audio into moving pictures. This can be considered as two separate problems:

• analysing the music to get useful parameters representing meaningful aspects of the music
  
  
• converting the extracted parameters into moving graphics that look impressive and that convey the character of the music to the viewer

Neither of these is easy, and just to make it that teensy bit harder for myself, I'm determined to do them in real time. I've been busy doing this for a couple of years now, and progress has been slow and jerky, but it has been progress. But one thing has always been a major problem: making the graphics look like they are 'organic' rather than computer-generated. Up until today, I had made some progress towards this goal, but the results were simple and lacked the complexity of things in the real world.






Just as with 3D, the problem is one of trying to represent the detail and diversity of the real world, but with very limited resources. Today I had an idea, and I've spent rather more time than I should have getting it working. The results are some of my most pleasing visualisations so far. Showing them is tricky, because they are generated at 1680 x 1050 pixels at 60 frames per second, and so capturing them is not easy, and produces very large files. I'm working on way to do the capture properly too...






So when you look at these screenshots, remember that they are stills from a moving animated graphics display, running in real time.


( The other thing to remember is that you can't use tables inside this blog application because it leaves large amounts of white space, and so formatting multiple pictures isn't easy. So I'll apologise now for the poor layout, which is mostly beyond my control. )

Drive-thru billboards...

I pre-ordered Criterion Games' 'Burnout Paradise' for the PS3 on the strength of previous versions of the game, plus the excellent down-loadable demo. Since then I've enjoyed many a happy hour driving extremely badly around a gorgeously detailed, non-existent place, bumping into other virtual cars, doing stunts that in reality I'd never attempt, and generally messing about in cars (and on motorbikes) in ways that you only normally see at the movies. Enormous fun!

You might have detected from my blog posts that I'm a bit of a completist, a perfectionist, a collector... Now those smart people at Criterion cater for many tastes in their video games, and they have got me sussed very well. Amongst several other 'collect these' challenges, is one where you have to drive through advertising billboards, or hoardings, as they are called in the UK. Note here that Burnout's Americanization is so complete that I now call them billboards, and that 'ization' crept in too. Not bad for a team of programmers from Guildford in Surrey in the UK: a county town more normally regarded as one place where stockbrokers live when they aren't working in the City of London.

There are 120 billboards to drive thru, and they vary from obvious, in plain sight, and easy to demolish, through to deviously hidden and virtually (!) impossible to reach. But I've managed to find them all and driven thru every one of them. I hasten to add that in the real world I've never done this, nor have I ever felt any need to do this, and I'm definitely not encouraging anyone to do it.

So you've got a game where people cruise around a city and its surrounding environs, looking for billboards... Sounds like a cue for adverts to me. And this is indeed true, because there are adverts for a number of well-known brands in the game. This is not that unusual these days, and I had more or less dismissed it as a way of making the virtual environment look more like the real world without descending to the fake adverts that you sometimes see that are funny at first, but soon start to pale.

So I was more than a little intrigued when I read that one of the contenders in the US presidential election had bought advertising space inside Burnout Paradise on the X-Box 360 in specific states in the US. In a delicious twist, you have something that shows very eloquently just how clever some people can be:

In a game where you drive around looking for billboards to drive through (thru), you don't/can't ignore the billboards!

Luckily for symbolism, the advertising billboards in Burnout aren't the same as the ones you can drive thru, but this doesn't change the brilliance of the concept. In most games with in-game advertising, you ignore the adverts in just the same way you ignore banner ads on web pages. But in Burnout Paradise, the game-play itself forces you to look at them. If I was speaking, I'd be lost for words in awe at this point, but luckily, I'm only typing.

I'm now wondering just where this leads next. Just how many ways can you weave adverts into the actual fabric of the game-play of the video game itself?

Strait talking...

Why is it a strait-jacket? Why not a straight-jacket - you know, something to keep you on the straight and narrow?

Pause here whilst you contemplate for a moment.

It turns out that it might be a straitjacket, presumably because you might dash away, or perhaps the hyphen could aid you in escaping from its confines... Which kind of gives away the 'strait' - it comes from 'restricted' or 'narrow' or 'tight', as in the Bering Strait, between Alaska and Siberia, or between the Pacific and Arctic oceans, depending on your point of view...

So is it strait-talking, as in confining your discourse to a narrow (an arrow?) range of subjects, particularly the truth?

Pause here whilst you contemplate for a moment.

Nope, turns out it is straight-talking, as in talking directly to someone and not getting distracted by either homophones or words that sound the same but have different meanings.

And as for strait and narrow, that is obviously a tautology, and so can be rejected strait away.

Downgrades as upgrades: Cartons

Over the last few years, I've grown accustomed to a strange phenomenon that has afflicted a once functional and pleasant land - the 'Downgrade as Upgrade'. I've watched in dismay as things that are supposedly new, improved, better, etc, have actually been impaired/restricted/reduced in features or performance. QuickTime has gradually lost the ability to convert between many formats like it used to, and seems increasingly to be just a player for the latest formats. Windows Media Player has lost just about all functionality beyond playing videos and adverts, and it has lost all its menus and usability in the process, but it has certainly gained the ability to stop me playing content that it thinks I shouldn't.

Just about every time I 'upgrade' a piece of software, I know with a heavy heart that this means that DRM will reduce my ability to use it, will further restrict what I can do, that features will vanish never to re-appear, that useful aspects will be twisted so they no longer delight me, and more. Much more. Trying to buy a monitor for my PS3 turned into a long, drawn-out battle trying to find out if an apparently suitable DVI-equipped monitor actually supported HDCP with the right version number. When LCD manufacturers seem determined to not tell you the specifications for their products so that you can check if the PS3 will allow itself to output a picture to the monitor, then I know the world has lost the plot (and any other idiom or euphemism). So how do you put something as big as the Earth in a strait-jacket?

And it isn't just software and hardware. The creeping malaise has spread much wider. Just recently, I've been trying to find a way to pour milk out of milk cartons without it going everywhere. Ten years ago, I remember that milk cartons came with a top not unlike a pitched roof, where you peeled apart the sides, and squeezed, then pulled one of the sides down to make a spout. Quick and simple, and it poured beautifully. Years of advances in science and technology later, and I'm faced with flat-top cartons equipped with tiny plastic widgets on top, where you flick open the tiny cover, peel away a foil cover, and then pour, and the milk proceeds to dribble down the side of the carton. In fact, it flatly refuses to pour properly.

I've asked around. No-one that I've talked to knows how to make them pour properly. Lots of milk gets wasted because you can't pour them properly. If this is such a great invention that it is on all cartons then why doesn't it work?

Why are we stuck with 'Downgrades as upgrades'? If I was cynical I might even suggest that politicians suffer from the same effect...

Hmmm. I seem to have written an article in a style that some people might associate with 'The Register'!

Hitting the Intellectual Wall

Athletes have their 'Wall': a psychological and physiological barrier that requires considerable effort to break through. Not being a particularly sporty type, I had not really thought that it might apply to other endeavours. But the recent emergence of a pattern (I'm definitely an 'emerging pattern' type of person - which is an interesting subject all to itself!) that I'd sort of glossed over made me realise that the concept might apply to a much wider scope than I'd thought.

The trigger was something I dabble with in rare idle moments: Sudoku. I was first introduced to it at a Wedding reception where they handed out sheets of brain teasers to occupy the guests whilst all the wedding reception stuff was being prepared, and there were two Sudoku taken from a broadsheet newspaper. Never having seen them before I solved them and thought that they were quite diverting. Since then I've got one of those electronic ones (naturally), and another one that has possibly the worst cursor-moving device ever perpetrated on human-kind, and several computer, iPod and iPhone versions.

The problem is, there's a sub-set of Sudoku problems that I can't solve. I reach a position where all of the remaining squares can be filled by one of a pair of numbers, and I can't figure out a way of proceeding past this block. I've reached my own 'Wall'. I've spent quite a while analysing the ones that I can't do to try and see if there's something I'm missing, but I haven't found it yet. Here's one:


Whilst I was racking my brain trying to solve it, I remembered a past situation with a close resonance to this one: when I tried to learn the 'Lisp' programming language. Now every other programming language I've learnt involved an initial feeling of being overwhelmed by the arcane difference between the newcomer and what I already knew, but eventually this was replaced with a moment of inspiration where 'I got it' and then things went as smoothly as learning a new language ever goes. This was how it was for Fortan, Basic, C, 68000 assembler, Pascal, HTML and Prolog...

But Lisp was different. Big Time different. 'You aren't going to like this' different. I unexpectedly hit the Wall and could not get past. And did I try! There was something there that just didn't compute. Just like this sub-set of Sudoku problems. Now I'm not saying that this isn't solvable, merely that I can't figure it out. Quite humbling, really.

So in my case, my own personal Wall is very tangible. Maybe I just need to keep metaphorically running, but it is certainly fascinating to see one close up and personal.

Value for money?

Many years ago, a very wise person said to me that in order to appreciate the true value of anything, you needed to compare it to something very tangible: a cheese sandwich, for example.

So when I recently bought a spare computer mouse to use when I'm presented with a laptop (I'm totally inept with a track-pad) I was intrigued to discover that it was cheaper than a cheese sandwich. Of course, other mice are available, and I'm sure that the same applies to cheese sandwiches too, but I didn't choose the cheapest mouse and the most expensive cheese sandwich I could find - merely representative ones from my bulk CD-R supplier and my local supermarket.

I'm not sure if the mouse or the cheese sandwich represents the best value for money...

This Japanese snack is good value for money - you get two rice crackers, one of which has a hole in it, plus the filling, all wrapped up in an individual wrapper. The real thing isn't quite so over-exposed of course, but it does serve to emphasize that this might be a trap for the unwary.

Actually this is a savoury snack, and despite the bright green wrapping for the complete pack containing about twenty of these, plus the writing which starts with the reassuring word 'chizu' (cheese or map in Japanese), the predominant flavour is actually mustard. Actually, they are delicious, and make a refreshing change from cheese sandwiches!

But I have no idea what the exchange rate of these for mice is!

Musically useful!

Every so often, I surf the internet looking for interesting music-related software. The things that I find vary from freeware with a wide range of qualities, through shareware which tends to be as good if not better than many more overtly commercial versions, to commercial software, where my rule of thumb is that the smaller and more dedicated the programming team is, the more worthwhile the software will be.

Some freeware is a by-product of academic research, and there are sometimes gems to be found here. It is all to easy to be put off by sparse documentation, or worse, documentation which points to a conference paper containing formal methods mathematics. But perseverance can reap huge dividends when you find something that is clever and useful, or maybe amazing and a stroke of genius.

One such marvel is ixiQuarks, which is 'hidden in plain view' here on the Apple.com download site. There's no need to feel guilty if you passed it by, by the way.

At this point I have to comment about an interesting use of words by the UK television channel called 'Channel 4', who responded to my query about why I could not view their TV programme online catch-up service on my Mac with a comment about them looking at ways of providing the service to 'non-Windows' computers - I had never, ever, thought of having a Mac or a Linux box as owning a 'non-Windows' computer! To try and balance this rather biased view of computing, I have to report that this software does not work on 'non-Macintosh' computers.


ixiQuarks is freeware from ixi audio, who describe themselves as: "an experimental project concerned with the creation of digital musical instruments and environments for generative music." This sounds rather like a music-oriented version of the very broadly-based 'arts and technology' collective that I'm a founder member of: the Curiosity Collective.

ixiQuarks, as the name suggests, contains a number of separate units of musical generation, processing, or utility. The first thing to do is to read the documentation, which means going to the bottom of the help file, and reading the section called 'Getting started: few studies'. This explains the importance of loading files into the 'BufferPool' utility, and provides help with setting up audio inputs and outputs. For me, it all went smoothly, and I was soon trying out the very neat user interface to the Karplus-strong algorithm generated sounds in the 'Quanoon' (presumably a clever play on words for the Latin phrase 'sine qua non' which means 'without which not' and which is interpreted as meaning 'an essential requirement', which is pretty accurate if you want a versatile plucked-sound controller.) instrument, and using 'SoundScratcher' to smear audio files using granular synthesis. Granular synthesis can provide a daunting array of controls, and so something that makes times-smearing simple and mouse controlled is exactly what I've been looking for - think of it as a bit like the Smudge brush in a bitmap paint program, but for audio.

But the highlight for me was rather played down in the documentation: 'ScaleSynth'. The user interface looks deceptively straight-forward: the screen fills with pale green rectangles. But moving the mouse around the grid and clicking (or holding the mouse button down) rapidly reveals the purpose as the rectangles go increasingly orange in hue: horizontal and vertical movements control pitch using a menu-controllable scale. It gives a marvelously simple and intuitive interface to controlling pitch, and I'm now wondering why more electronic music software does not use this type of front end. This is probably my best 'find' for this year, although Puremagnetik's Vector is a close second.