Monday, 26 June 2017

What is the scam here?

We have a (non) customer the keeps trying to buy a mobile number to receive texts.

Our automated credit control systems trap him every time! He makes new postal addresses, email addresses, bank details, telephone numbers, etc, but the systems are working well. The latest one flagged the account in 15 different ways!

Every time, he orders a mobile number and has some system send him a text from "AUTHMSG" to him which looks like Your Valued Opinions verification code is: XXXXXX.

He does not get the text due to the account being flagged as suspicious.

Obviously he wants a UK mobile number to get some code to do something, but I am at a loss as to what the scam is here.

I tried googling but still did not find what the scam is...

It is annoying the accounts staff, reversing out the account and invoices. 29 attempts so far!


Treading on thin ice? I am MD of a communications company - I (and the company) have to respect the Data Protection Act, and obviously as a company we take things very seriously, not just in terms of law but in terms of morals and ethics. In this case we have someone I cannot really identify as a person - but I have managed to identify (correlate) multiple fraudulent attempts as probably the same person that is trying to act illegally - so is there personal data? I posted the slightly redacted content of a message, is that personal data? Even so, there are exceptions in the legislation for prevention and detection of crime, so is that valid? It is a good point and maybe a fine line, on which I hope I am the right side.

At the end of the day, if the individual in question feels aggrieved, I am more than happy for them to identify themselves and raise the issue with me or the company. There are several outstanding invoices as well as possible criminal charges for fraud and breaches of the Communications Act that await if you feel I have breached your privacy - please go ahead!

Thursday, 22 June 2017

Badly written RFCs

There are many badly written RFCs, but I encountered one annoying one today

Update: Thanks for the comments, and I concede that the next page refers to "value" being quoted. Even so, the whole idea of using BNF syntax is to be unambiguous, and this is still therefor a rather badly written RFC!

So, for those less techie, "RFC" is "Request for comment" which is a sort of passive way of pushing proposals on other people when the Internet first started. The "standards" we now follow are all RFCs. There is a slightly more formal process that can promote an RFC to an actual standard.

The idea is that the RFC says how something works, especially when it is a protocol. People they try to make their systems work to the RFC.

Now, there is always a degree of ambiguity, and so there is a really good principle which has worked well for the Internet which is that you should be tolerant of what you receive and strict in what you send. Basically, if the standard says to do something you should aim to be as accurate and correct as possible in what you send. However, when someone sends something to you, and there is some flexibility in what you accept, you should try to be flexible and work out what it means.

This has allowed the slightly flawed and imperfect implementation of many standards.

Today's issue was an RFC over MIME email, specifically RFC 2387 which apparently has status of "proposed standard", and is 19 years old.

We were sending a MIME object in an email that is multipart/related with a "type" field, specifically type=text/html which says that the "root" of that object is of a type text/html. All well and good. Indeed, a type= attribute is mandatory in RFC2387.

The problem is that the email did not work properly when emailing yahoo addresses. But other email systems did work. We experimented and found the "fix" was to send type="text/html" instead.

Now, I am not happy about this! The RFC has this to say on the type attribute :-

This defines that type is specified (after a ;) has the text type, then the character = and then the type and the character / and then the subtype.

So, unless we are saying that the type is "text and the subtype is html", sending type="text/html" is simply wrong.

The problem is the examples in the RFC (and errata), such as :-

(the errata added the missing ; on the end of the first and third line)
Where the hell did those quote marks come from?

So, yahoo are not being tolerant in what they accept, they are actually expecting non standard data, and we were being strict in what we sent, but to make it work we are now being non standard.

There really is not much worse than an RFC where its own examples do not comply with the RFC!

I have submitted and errata to the RFC editors for this.

Wednesday, 21 June 2017


It turns out that the algorithms for creating a maze are quite diverse and complicated - wikipedia has much on this.

Basically, I have done it again - I found something that looks cool on thingiverse and decided to take it to the limit and do it myself. The thing was a simple labyrinth box. It was quite cool.

So I thought I would give it a try, and the issue is not really the actual 3D artwork and OpenSCAD stuff, it is the maze itself. How to make a maze that is challenging. Instead of a fixed thing that has a maze, I wanted a random maze so each one is different. OpenSCAD cannot quite manage that. So I used the source (Luke) and wrote C to make OpenSCAD.

The first thing is that this maze inherently wraps, i.e. mostly a maze is in a simple rectangle, but this is on a cylindrical shape so the X axis wraps. Easy enough for the maze generation logic.

My initial thought, and how I have made mazes before, is you have a path from origin and a point moves randomly where it can (i.e. to an empty cell). If it finds it cannot move at all it back tracks. This is a very simple algorithm to fill the whole space of a maze with no loops. The "no loops" part is pretty common as a basic principle. Maybe I'll deliberately throw in a loop some time.

My concern was that this made a maze that was going to be too simple. Or that did not "look nice", so I added a few variations.

For a start, when moving, I made a bias to continuing in a straight line rather than left or right. 25% of the time it continues, else it is random (including continuing). This makes for nice long runs in the maze.

I then added a bias for back-tracking even when not a dead end. This did make the maze more interesting but created a lot of one unit blind dead ends which are annoying and boring. You want your blind dead ends to be "interesting" and so long.

So I started again and made it that a set of all cells in the maze with somewhere to go, are picked from at random to continue the maze. This is not following the last point and backtracking, it is saying we have a set of points that can move, and picking one and moving.

This created a lot of short blind paths, hmmm... OK next trick was to add a random bias to use the last added point and continue from there, but not all of the time (just 75%). This made much longer blind runs which is what I wanted. I actually made it wander like this 100% initially until it hit a dead end or the top layer, then 75% leaving 25% of the time it picks any node at random to continue from making a fork.

Finally the trick was to then make the exit at the top the cell where it is the longest path from the start. This helped avoid accidentally having a straight line from start to finish or something that simple.

I am actually quite pleased with the result, which you can find at thingiverse. It is interesting how slight biases and choice of algorithm can massive change the nature and appearance of a maze.

P.S. You will note that there are a number of example STL files on that thingiverse entry. Obviously, as I have been tinkering, changing the artwork slightly, and changing the maze design logic slightly, I have wanted to update the thingiverse entry. But this is a set of about 10 designs as examples, so a tad tedious to load each one in to OpenSCAD and make an STL file. Thankfully OpenSCAD (even on a Mac) has a command line option, so with a simple script (and waiting several minutes) I can make a set of files automatically!

P.P.S. I did do a bit more tinkering, not just the artwork (changing from round to polygonal outer shape), but also to the maze. The one "knob" in the lid was not that good so I made the maze mirror on to the other side so allowing two knobs - this simply means as I generate the maze I add the same to the far side, but the maze can still going around the cylinder interacting with its other self even. I made it general to allow N paths. I also tweaked the bottom (end point) of the maze to be a right angle. I even scripted rendering several random boxes to make a video :-

Update: I had a call to make the maze more complex. One of the simple steps I did is, as you progress the maze, for each cell, you also have an indication of "length" which is how far from the start, increasing one each time. Then, for the exit at the top layer I pick the point with the greatest length.

However, this is not quite enough, and so I changed the logic to start the maze in the middle, and pick both the entry and exit as longest from that point. Unfortunately this allowed a common path from the start point and then a short path to entry and exit. But it did seem to make a more interesting maze overall.

The solution is creating the maze then working out the start point and exit point that create the "best" path. I decided simple length is not good enough, we want complexity. Scoring is therefor also based on how many branch points we pass as well as length.

I also made the middle a ring all the way around on the taller mazes to make a staging point where you had lots of possible options to escape, all but one of which are wrong.

Tuesday, 20 June 2017


So, we have a case of someone feeling they should mow down a group of muslims praying, and why? Presumably because it was muslims that did something nasty last time.

This logic is terrible, and if you follow the same logic forward, the logic used for all race and religious hatred, you must now panic at the sight of a white van man!

On my drive right now, we have two of them (carpenter and decorator), so that must be terrifying to the neighbours!

If people can see that such a comment makes no sense, how can they not see that such comments about muslims or any other race or religious group are just as daft.

Monday, 19 June 2017

Controversy of empty London buildings

After the Grenfell Tower fire there have been interesting questions raised in the news, and these are seen as Labour vs Tory, but are not so simple I am sure.

The issue, it seems, is there are many empty houses in London for various reasons, and there have been some calls to force those to be used to house people stranded after the fire.

Before I am start on my musings on this issue it is worth pointing out that this question should not have come up, surely?
  1. It should not have happened - no way a tower block should catch like this - we have building codes, and health and safety nightmare legislation for a reason.
  2. Why can the council simply not cope - it is a building managed for the council - but there has to be some fire insurance, and anyway the council must have contingencies to take care of people in an emergency, surely? The council can pay for people to be in hotels if needed. The landlord should surely have some responsibility for this?
But, rants aside, my thoughts here are really on the issue of empty houses, and the idea of commandeering them to house people in this emergency.

Owning property is a right!

The issue comes down to basic rights to own property. If you own a house, whether you are living there or not, you have rights. We have basic concept of property rights in this country. These will always be unfair by their very nature - i.e. if I own a thing (whether an empty house in London or a Mars bar) and you need it, but cannot afford to pay for it, then that is a problem. If you are starving and I have a Mars bar, do I have a moral obligation to give it to you? Probably, unless I am starving, or I was going to give it to someone else who is starving and giving it to you would deprive them. Ethics and morals are hard, aren't they. Even so, however unfair property rights are, they do exist, and we do not want to abandon them on a whim or even an emergency like this.

So what does it mean? Well, we do have laws that curtail some property rights. There are rights over monopoly control of things, where people can set any price they like as they own all of something. There are rules for landlords even when they own the properly (rules that should make them responsible for helping tenants out in cases like this I am sure). But we only curtail these basic rights in very rare cases. Is this such a rare case where new laws should apply?

Emergencies only?

One thought is some sort of emergency measures arrangement - where something like this would be some sort of special case. But that is complicated. There are around 600 homeless after the fire. So is 600 homeless people an emergency deserving a law allowing homes to be commandeered? Seems legit, until you realise that "Shelter calculates that 170,000 people are homeless in London today" so if 600 is an emergency we would have a permanent emergency and nobody would want to have an empty house in London. Indeed, anyone investing in London property would take down the house, or make it uninhabitable, so they avoid this special emergency legislation. Such a law would not actually help...

Of course, if you do allow houses to be taken over, how far do you go. Does everyone with an empty bedroom have to take on an otherwise homeless stranger? Where would you draw the line?

But some people want to help!

This is good news, there are builders that have finished flats and houses about to go on the market but offering for free as an interim measure. Apparently this is an issue, as anyone taking such a place apparently gets taken off the council housing list to find somewhere (as they have somewhere now!). This is bad for them, and the person offering the temporary accommodation.

In fact, it is hard to offer temporary accommodation without issues with sitting tenants or squatters that you cannot get rid of. Normally this is good for tenants - having strong laws to help them. But this is probably the very reason why there are lots of empty houses. If renting them out was a "safe" thing to do, people would do it and make more. Sadly, if they do, then selling their investment property is harder - they have to find someone prepared to take over the tenant or some way to get rid of them, so simpler to keep the place empty. There is, perhaps, the real problem here.

Maybe we need some type of tenancy where you have a fixed term and an absolute right to evict at the end of that time, come what may, but not a moment before. That would allow a very clear cut arrangement, a house for 3 months, but not a day more unless you are offered an extension. Safe for landlord - no messing with court orders, etc, just evict - and safe for the tenant as he knows where they stand and has security for that fixed period.

Of course, if such tenancy agreements existed, they would be used all the time, so that is not good. Maybe an arrangement like this could work where there is zero rent. People would only offer such arrangements in emergencies like this, but would not be hindered, or stuck with squatters, if they do offer generosity like this.

To be clear here, I am not saying that people will stitch up someone being kind, but they may find themselves with no choice. If you are homeless today, offered a house for 3 months, and then find the only option is being homeless again you may choose to become a squatter if that is allowed or at least prolongs your stay. If the alternative is being homeless for those 3 months instead, surely it is better to have someone prepared to offer somewhere, even on a short term arrangement like that?

Is there a solution?

The issue is people feel it really unfair that there are so many empty houses, and so many homeless (not just because of the fire). Maybe the existing legal framework is skewed just a tad too far towards the tenant? Maybe if there were some changes that could make it sensible for people with empty houses to rent them out then we would not have empty houses. It may simply be a case where the law has pushed too far in favour of one side of a commonly adversarial contractual arrangement (tenancy) to the ultimate detriment of both parties, and hardship of potential tenants.


Needless to say, I once again think the issue is not the symptom we see (empty houses) but what legal and economic framework resulted in that being a sensible thing to come about. That is what needs fixing, not the symptom. People need to "take a step back" a little more often.

Sunday, 18 June 2017

Much better motion blur using Final Cut Pro

Thanks for the feedback on the last blog post.

I agree, it looked way too ghosty, and I have finally worked out why.

If you want to do this, you need to be a bit careful with the settings, and also a bit careful with believing what people say on the various Final Cut tutorial videos.

You use Motion to create a Final Cut Title.

You then do a few simple steps:-

  • Remove the text element - we are just showing the "background"
  • In "Render", click "Motion Blur"
  • In "Projects", "Inspector", "Properties", set motion blur samples and shutter angle.
  • Save it with a suitable name for Final Cut to see it as a title effect

Some experimentation confirms what I understood. I am applying this to a sped up video. In my case 60x normal speed. So I set 60 samples and 360° shutter. What this does is for each frame of output it merges all 60 frames of original creating a smooth motion blur effect in that one frame the same as if this was actually a 60/25 (2.4s) second exposure for that one frame. This is the effect I actually wanted.

So what did I do wrong?

Firstly the various tutorial videos say the duration of the title effect does not matter, so I made a 10 second (the default) duration effect. I placed it over my (sped up) video, and stretched to fit (in this case, 3 minutes).

What I did not realise is that stretching the effect meant that it stretched the sampling. Each output frame was indeed merged from 60 input frames, but the sampling time period had been stretched 18 times, meaning that it spread over not 2.4 seconds of original (which makes one 1/25th second output frame) but over 43 seconds of original, moving 2.4 seconds forward each frame. Hence the rather odd ghosting effect.

The solution was to make an effect that was 3 minutes long and not to stretch it!

Is there another way?

One of the tutorial videos said to set the effect to loop one frame before its end. Presumably to make a 10 second effect, that when stretched, did not stretch the sampling.

It worked for the 1st 10 seconds, but unfortunately this simply meant the video looped every 10 seconds.

Maybe there is some other setting to define the way the title effect works when stretched, I may have a play.

How did it look?

Here are the three videos again.

Saturday, 17 June 2017

Time lapse and motion blur (Final Cut)

I have a slight dilemma with my new camera. It can record for about two hours because (a) the 256G card I have records about that and (b) the battery lasts about that long.

So, if I want a time lapse I have to change battery every 2 hours or so anyway, so may as well change memory card. Hence not a lot of point in recording in "time lapse" in the first place unless I can connect some mains power to the camera. There may be cases where that is sensible and the time lapse feature in the camera makes sense.

However, for now, I figured I may as well record in real time, and then make a time lapse in Final Cut Pro. It is very simple to tell it to speed up a clip.

There is a down side, the time it takes to transfer and process 256GB of data at a time, and so on. However, I think Final Cut Pro could do a better job than the simple time lapse.

The first thing that occurred to me is that the time lapse would be good if it did a proper "motion blur" on the images. So if I am doing 60x time lapse, process 60 frames to make a motion blur for each frame.

It turns out that whilst this is not a native feature of Final Cut Pro, it is a feature of the titling functions. So by buying the "Motion" system, and creating a text title with no text in it and set to do motion blur as a feature, I can overlay that on the sped up video. Yes, it is complicate to make it (and you can google for videos of how to do it), but it works.

Down side it that it takes a while, well, especially if you do 60 samples at 60x speed like I am. It takes a very long time! I need a faster machine (roll on December, Apple).

But the result is impressive.

So first the non motion blur video - which is as you might expect - still very good :-

And now the motion blur version :-

As you can see, quite a difference. It makes for a very different time lapse, and looks ghosty - see next post for the fix!

So, when the do eventually start building the actual climbing frame, etc, I should be able to make a nice time lapse.