Coding Challenge Results

[Dr. VI]

These should be automated and going to the web site
within the next week or so, but here is a hint at the current
results.

31 Craig Smith *
34 Bart Michiels
35 Martin Gustafsson
38 Philips Research, T&MA team
44 Teddy Roerby
49 Remco Breen
51 Greg Falcon *
53 Greg Falcon *
59 Craig Smith *
64 Remco Breen
66 Philip Carmichael *
...

The number is how many milliseconds for the VI to
decode an array of various datasets. * by the name
indicates that the person has been modified and needs
to be saved -- wait, it means that they work for NI and
are ineligible for winning the competition. They are
simply there for the fun of it.

The VI was run on several P3 and P4 PCs. The times
varied quite a bit, but the rankings didn't c
hange that
much. The times above are from a 2GHz P4.

Overall, the submissions have been quite interesting,
and I think that we will soon start publishing the
techniques that distinguish some of the better
submissions from the rest of the pack.

I am not ready to post a list of submissions that
produced incorrect results, but I will review the intent
of the challenge rules.

The official encoder will produce some characters
in the input string with invalid parity. It will check to
see that the decoded string has the original characters
for those with no parity problem, and it will check that
the data with incorrect parity was decoded as a '?'.

So, if you are pretty sure that your VI should be in
the running, but it isn't on the list subset shown above,
that may mean that it failed to correctly handle parity
checking. When this is uploaded to the web site, this
will somehow be more clear.

If you haven't already spent enough time coming up
with the best solu
tion, the doors are still open.

[Ben]

I don't get it!

Where are the sub-ms runs I read about?

Am I missing something here?

Ben

[dcwork@hotmail.com]

Hello,
Please give the ms according to the longest string in the given program, so that I can compare the result and decide if I need submit my result to you or not.

[altenbach]

I think there are two points to be made.

(1) This test has nothing to do with the supplied strings, but loops through an array of various strings, some with parity errors, and check for correct operation. (Can we have a list of all string lengths used in the set?)

(2) Several users have multiple entries. Apparently, we can submit as many as we want and the fastest for this particular scenario will be used. Just submit your fastes solution, and you'll see how you stand.

[Greg_McKaskle2]

> (1) This test has nothing to do with the supplied strings, but loops
> through an array of various strings, some with parity errors, and
> check for correct operation. (Can we have a list of all string lengths
> used in the set?)
>

Personally, I still think that you need representative samples just to
get feedback on correctness and which of two approaches is faster. I
don't think that designing the algorithm for a particular dataset is the
right approach.

> (2) Several users have multiple entries. Apparently, we can submit as
> many as we want and the fastest for this particular scenario will be
> used. Just submit your fastes solution, and you'll see how you stand.

Actually, all your submissions will be used. The multiple submissions,
at
least the internal ones all resulted from someone's claim that "this
solution is optimal, can't be improved upon", and then the next morning
they would submit another one with an improvement. I think that five
iterations is the current record.

Greg McKaskle

[Greg_McKaskle2]

> Where are the sub-ms runs I read about?
>
> Am I missing something here?
>

The sub-ms runs are for a particular dataset, the long one that came
with the encoder, and running on particular computers. The times in the
rankings are for a different dataset running on a different computer.

I was intentionally making the dataset larger, and making more datasets,
instead of just iterating over the same dataset. Why? Because that is
more like the real world, and some people were writing caches to compare
this buffer to the previous so they could just return the last answer,
thereby benefiting greatly from the iteration timing approach.

Greg McKaskle

[Greg_McKaskle2]

> Please give the ms according to the longest string in the given
> program, so that I can compare the result and decide if I need submit
> my result to you or not.

Because a 300MHz PIII and a 2.7 GHz P4 are almost a factor of ten in
clock speed, I don't think that it is reasonable to compare them this
way. Feel free to compare with others on the list, but the official
times, at least until the competition is over, will be made on the
official dataset. If you have a solution, what do you have to lose?
Submit it.

Greg McKaskle

[Ben]

", what do you have to lose? "

Time!

If my code is in the running, it will be worth submitting. Otherwise, I will just wait to see what I can learn from the others.

Ben

[Greg_McKaskle2]

> ", what do you have to lose? "
>
> Time!
>

I suspect that these email's have taken more time than actually posting
the results. Of course that is just my guess, and I'm also biased to
getting lots of good submissions.


Greg McKaskle

[altenbach]

> I don't think that designing the algorithm for a
> particular dataset is the right approach.

Sorry, that's not what I meant. 😉
I just wanted to get an estimate of CPU cycles per processed character.

(Currently, I am testing with a set of 10 randomly generated strings from 10k to 1M length using a test VI that sequentially tests an entire folder full of candidates (via call by reference node), picks all correct solutions, then sorts them by speed. I don't think it is possible to "design" for a particular set, the results don't change much if I e.g. divide all lengths by 100.)