speak, it ain't illegal yet

April 20, 2007

“You’ll be arrested”

Filed under: chicago, ethics, photography, police, rights — diginux @ 10:21 pm

Today was the first time those words had been spoken to me, at least the first time from someone that could actually arrest me. Today my fiance got an email from her school saying there was a shooting for an episode of the TV show ER in the park next to her school and they were looking for extras. We had plans earlier in the day, so we couldn’t get there in time to signup to be an extra, but decided to go in order to hopefully get a few fun shots of a live taping of a TV show.

When we first got there, we were walking on the sidewalk and decided to walk up a little closer to see what everything looked like. We were quickly stopped by a friendly security guard who said we had to stay back from the set. That was no big deal to us, so we kept on walking around the set to the other side. We were a good 100 feet from the set, and no one seemed to care that were watching. That is when I decided to take out my camera and shoot a few pictures.

Almost immediately a Chicago Police Department officer started walking towards us. When he got within talking distance, he immediately started saying I needed to leave immediately or I would be arrested and if I decided to come back, I would also be arrested. I politely asked him what I would be arrested for, he gave a monotone reply of, “Copyrights.” Then he quickly said, “So, leave.”

I know very well that what he said was a lie. In general photographers enjoy the right to photograph almost anything if they are in a public area. However, I didn’t feel like starting an argument with the officer, as I have good reason to not want to be arrested, including the fact that I had my fiance with me, so I left it at that and left.

Just to make sure I was in the right, I decided to ask one of my favorite photography forums. It appears that I indeed was in the right. However, if anyone reading this is a lawyer, or knows a lawyer, and could find a specific answer, I would appreciate it.

I am still steamed from whole experience. I really dislike it when people with power abuse that power with intimidation and lies. Further, no matter the circumstances, it is never easy to knowingly not put up a fight when your rights are being infringed upon. I can only hope that this is an isolated incident.

April 17, 2007

Announcing Chicago GNU/Linux User Group Video Archive

Filed under: chicago, chiglug, gnu, gnulinux, linux, lug, video — diginux @ 8:58 pm

I have just put the finishing touches on the Chicago GNU/Linux User Group Video Archive. At the archive you can both play videos through your web browser as well as save them to your computer. The videos are licensed under the GFDL and freely distributable under those terms. The first three videos are from earlier in the year. I plan to start video taping every meeting that I make. Hopefully in the near future I will also be able to start streaming the video live from the meetings.

If anyone has any ChiGLUG related videos I would be more than happy to put them up. If anyone has video from Flourish, I would also like to put that up.

If anyone has any comments or suggestions for the video archive, please post a comment. Thanks.

April 3, 2007

Writing a duplicate file finder in Erlang

Filed under: duplicate files, edupe, erlang, programming — diginux @ 2:38 pm

A couple of days ago I decided I wanted to go through my collection of music in order to get rid of duplicates. I would have assumed that this problem had been solved already, but when I googled “linux file duplicate finder” the results were not so great. The first result was a program called FSlint. It wasn’t in portage, and I didn’t feel like finding out what its dependencies were, so I skipped to the next result, which was someones duplicate file finder that they wrote in Java. It was amazingly slow.

Therefore, I decided, what the heck, I will just write my own in Erlang! It sounded simple enough, just get a complete file listing, sort the files by their file size, then go through and compare files that have the same size(since a different file size would mean a different file), if you find files with the same size, get the MD5 sum, then compare those.

It turned out to be slightly more complicated than that. First of all, the filelib:fold_files function gets stuck in an infinite loop if you have a symlink that points to a higher parent directory in which that symlink is contained. Wine actually does this when you set it up. So I had to write my own function to generate a file listing that just ignores symlinks for now.

Another thing I found out was the amazement of how many files I had that were the same size. Taking hashes of all these files is a pain, especially if they are large. So I decided I would instead just take the first 256 bytes and the last 256 bytes of the file and hash that. If the file was less than 512 bytes, I just hash the whole file. Using this method increased the performance by 400% and maintained the same accuracy, though is technically still a small possibility of a false positive, but it is good enough for what I need.

The funny thing was, after I was almost done, a friend on the #chiglug@irc.oftc.net channel referred me to a perl program called fdupe. I tried it out and found out it actually did what I wanted and used close to the same methods. So I decided I would benchmark it against my implementation.


For a collection of 35,042 files that totaled 168GB, my implementation was 23.77% faster
fdupe: 2m23s
edupe: 1m49s

For a collection of 220,463 files that totaled 85GB, my implementation was 16.46% faster
fdupe: 35m39s
edupe: 29m47s

It is good to see that Erlang can hold its own.

I have released the code under the name EDupe, and you can download the latest code here.

Blog at WordPress.com.