02.02.07
Suddenly, all your history's ablaze
Try to breath as the world desintegrates
Just like autumn leaves we're in for change
Holding tenderly to what remains
And all your memories are as precious as gold
And all the honey and the fire which you stole
Have you running through all your red cheek days
Shaking loose these songs from their sacred hiding space
Hold your heart courageously as we walk into this dark place
Stand, stare fast, erect and see that love is the province of the brave
Push under the ?(whizing lights) of buzzing stars
Let this burning brightly illumintate where we are
Build this hallow that lovers voices occupy
Let it follow
That we let it free, let it fly
Breaking open the walls of this cage
Intoxicated, oh so amazed
Much like falcons tumbling from our hearts that blaze
Oh come join, ?(arms engaged)
Hold these hearts courageously as we walk into this dark place
Stand, stare fast beside me and see that love is the province of the brave
"Province" by TV on the Radio
Try to breath as the world desintegrates
Just like autumn leaves we're in for change
Holding tenderly to what remains
And all your memories are as precious as gold
And all the honey and the fire which you stole
Have you running through all your red cheek days
Shaking loose these songs from their sacred hiding space
Hold your heart courageously as we walk into this dark place
Stand, stare fast, erect and see that love is the province of the brave
Push under the ?(whizing lights) of buzzing stars
Let this burning brightly illumintate where we are
Build this hallow that lovers voices occupy
Let it follow
That we let it free, let it fly
Breaking open the walls of this cage
Intoxicated, oh so amazed
Much like falcons tumbling from our hearts that blaze
Oh come join, ?(arms engaged)
Hold these hearts courageously as we walk into this dark place
Stand, stare fast beside me and see that love is the province of the brave
"Province" by TV on the Radio
23.01.07
It's so so true. This is what I read over at Joels.
Now there is more you need to read before the above paragraphs to get a fuller understanding.
That stated, this kind of thing bit me in the arse this weekend. In the job I have now, I am learning a long list of new things (some of which are very easy) as I go. That long list includes Ruby, Ruby on Rails, Test Driven Development, and Agile development. It also means that some things that shouldn't take very long at all for programmers with lots of experience in a given environment are killin' me!
Friday when I left work with a Monday deadline for some new functionality I knew two things.
So confident was I that this stuff really wasn't a big deal, I didn't start Sunday morning like I had originally planned. Instead, I stayed Saturday night at a friends house after a show, had lunch, then didn't start working on it until 3 in the after noon. So sure was I that everything was under control, I even allowed time to chat with a particularly nice young lady sitting next to me with jet black hair and eyes like blue ice that bored right into you.
I'm drifting.....
Well, as I got down to business, the difficulties soon made themselves known and things became darker and darker. And what was the jist of the problems? Ultimately it's the fact that I'm still too new to Ruby to just be quick about it. A couple of things would've helped big time. As a matter of fact, my biggest gripe after this experience is with Pretty Print. Now I guess you can customize it's output, but I'm a noobler in Ruby so that's not going to happen while I'm under the gun!
The problem for someone that is accustomed to using PHP and it's various ways to dump raw data to the screen is that Pretty Print doesn't format the data in some nice way and it doesn't state if you are looking at a hash or an array. This alone caused me no end of heart ache. About 4 hours worth as a matter of fact. Once I figured out that the data set I was getting back was a hash as opposed to an array, a quick "to_a" sorted out all the drama and things started to look real sunny.
Now before you get started, I know what a hash is and how it's different then an array, but trust me when I say that the output from Pretty Print doesn't make that plain.
Anyway, the last two days of my life has been garbage! Tired as hell from 12+ hours of bumbling around in the dark writing code, the resulting lack of sleep, the 8 gallons of caffeine needed to stay awake through the day, my turbo car suffering a bearing failure, and my ex telling me that she might be pregnant. There's too much stuff going on and I've never been good at juggling.
And so far, I don't like Ruby!
One of the unfortunate side effects is that your mind gets into a bad habit of overestimating how clearly it understands things. It always thinks it has The Big Picture even when it doesn’t.
This is a particularly dangerous trap when it comes to software development. You get some big picture idea in your head for what you want to do, and it all seems so crystal clear that it doesn’t even seem like you need to design anything. You can just dive in and start implementing your vision.
Now there is more you need to read before the above paragraphs to get a fuller understanding.
That stated, this kind of thing bit me in the arse this weekend. In the job I have now, I am learning a long list of new things (some of which are very easy) as I go. That long list includes Ruby, Ruby on Rails, Test Driven Development, and Agile development. It also means that some things that shouldn't take very long at all for programmers with lots of experience in a given environment are killin' me!
Friday when I left work with a Monday deadline for some new functionality I knew two things.
- I figured out the cryptic interface for the module with no documentation.
- The formatting of the generated information would be no big deal as it's using the same partial anyways right?
So confident was I that this stuff really wasn't a big deal, I didn't start Sunday morning like I had originally planned. Instead, I stayed Saturday night at a friends house after a show, had lunch, then didn't start working on it until 3 in the after noon. So sure was I that everything was under control, I even allowed time to chat with a particularly nice young lady sitting next to me with jet black hair and eyes like blue ice that bored right into you.
I'm drifting.....
Well, as I got down to business, the difficulties soon made themselves known and things became darker and darker. And what was the jist of the problems? Ultimately it's the fact that I'm still too new to Ruby to just be quick about it. A couple of things would've helped big time. As a matter of fact, my biggest gripe after this experience is with Pretty Print. Now I guess you can customize it's output, but I'm a noobler in Ruby so that's not going to happen while I'm under the gun!
The problem for someone that is accustomed to using PHP and it's various ways to dump raw data to the screen is that Pretty Print doesn't format the data in some nice way and it doesn't state if you are looking at a hash or an array. This alone caused me no end of heart ache. About 4 hours worth as a matter of fact. Once I figured out that the data set I was getting back was a hash as opposed to an array, a quick "to_a" sorted out all the drama and things started to look real sunny.
Now before you get started, I know what a hash is and how it's different then an array, but trust me when I say that the output from Pretty Print doesn't make that plain.
Anyway, the last two days of my life has been garbage! Tired as hell from 12+ hours of bumbling around in the dark writing code, the resulting lack of sleep, the 8 gallons of caffeine needed to stay awake through the day, my turbo car suffering a bearing failure, and my ex telling me that she might be pregnant. There's too much stuff going on and I've never been good at juggling.
And so far, I don't like Ruby!
15.01.07
Programming sure is a bummer these days! [Rants and Rambles] -
BDKR - bdkr@highsidecafe.com @ 14:53:46
And it's not the programming that's bad. It's the environment. Other coders and the tribal/religous side of it. My last job hunt and recent things I read over at Loud Thinking have really driven this home for me. For the sake of brevity, let's just talk about Loud Thinking.
Joel Spolsky recently posted a piece called "Language Wars" and really managed to step on some toes from those that drink the koolaid: the Ruby Devout. Of the things that he said, there are certainly a couple of points that I agree with in particular.
So while the above statements made the hordes unhappy and the pitchforks came out, there really is some truth in the things he had to say there. But instead of challenging him on those truths, they chose to call it FUD and be done with it. The really interesting thing is that he never said he didn't like the language. What he said amounts to "it's bleeding edge right now and it's prolly not to smart to use in in an enterprise setting", to which, I have to agree (but I also think it's cool if are choosing to be on the bleeding edge!).
Once again, there was no answer for this, but instead, "It's FUD and he's a jerk".
But the truth is that there are scaling issues to be sorted. And there are performance issues. On the topic of performance issues, it's not with Ruby itself in my opinion, as it's most likely "fast enough", but with Rails and it's use of ActiveRecord. ActiveRecord is an Object-Relational Mapper layer with the aim to abstract typical DB usage away as much as possible from the OO developer. Models obviously become very important keeping this in mind.
Anyway, all an object relational mapper is going to do is suffer the drawbacks of what OODBMS'ers call "an impedance mismatch". In other words, the overhead of developers trying to extend their object like hierchal view of data onto the database itself. In this case, retrieve the data then manipulate into something the developer wants to see. Now I wont argue the merits of wishing to view and deal with data in this manner, but the performance hit can't be overlooked.
That said, Rails has a big issue and it's very core: the performance of it's ORM.
The bigger problem for Ruby and all other languages with people developing Rails like frameworks with ORM's at their cores is actually bigger then the use of an ORM in the first place. It's the idea from zealous adherents that this is OK and that no real thought needs to be given to it. They've chosen to abstract the DB away so much behind various layers of OO fer-dizzle that most don't even know the term "impedance mismatch" or have never even heared of an OODBMS. So busy bowing before an alter and burning heretics, they seem incapable of realizing that using an OODBMS could solve the performance issues in one fell swoop.
But what do I know eh?
Joel Spolsky recently posted a piece called "Language Wars" and really managed to step on some toes from those that drink the koolaid: the Ruby Devout. Of the things that he said, there are certainly a couple of points that I agree with in particular.
- ...it's known to be slow, so if you become The Next MySpace, you'll be buying 5 times as many boxes as the .NET guy down the hall.
- I'm really not sure that you won't hit scaling problems, or problems interfacing with some old legacy thingamabob, or problems finding programmers who can understand the code, or whatnot. So while Ruby on Rails is the fun answer and yes I've heard of 37 Signals and they're making lovely Ruby on Rails apps, and making lots of money, but that's not a safe choice for at least another year or six.
- ...I'm sure you'll have a lot of fun, but for Serious Business Stuff you really must recognize that there just isn't a lot of experience in the world building big mission critical web systems in Ruby on Rails.
So while the above statements made the hordes unhappy and the pitchforks came out, there really is some truth in the things he had to say there. But instead of challenging him on those truths, they chose to call it FUD and be done with it. The really interesting thing is that he never said he didn't like the language. What he said amounts to "it's bleeding edge right now and it's prolly not to smart to use in in an enterprise setting", to which, I have to agree (but I also think it's cool if are choosing to be on the bleeding edge!).
Once again, there was no answer for this, but instead, "It's FUD and he's a jerk".
But the truth is that there are scaling issues to be sorted. And there are performance issues. On the topic of performance issues, it's not with Ruby itself in my opinion, as it's most likely "fast enough", but with Rails and it's use of ActiveRecord. ActiveRecord is an Object-Relational Mapper layer with the aim to abstract typical DB usage away as much as possible from the OO developer. Models obviously become very important keeping this in mind.
Anyway, all an object relational mapper is going to do is suffer the drawbacks of what OODBMS'ers call "an impedance mismatch". In other words, the overhead of developers trying to extend their object like hierchal view of data onto the database itself. In this case, retrieve the data then manipulate into something the developer wants to see. Now I wont argue the merits of wishing to view and deal with data in this manner, but the performance hit can't be overlooked.
That said, Rails has a big issue and it's very core: the performance of it's ORM.
The bigger problem for Ruby and all other languages with people developing Rails like frameworks with ORM's at their cores is actually bigger then the use of an ORM in the first place. It's the idea from zealous adherents that this is OK and that no real thought needs to be given to it. They've chosen to abstract the DB away so much behind various layers of OO fer-dizzle that most don't even know the term "impedance mismatch" or have never even heared of an OODBMS. So busy bowing before an alter and burning heretics, they seem incapable of realizing that using an OODBMS could solve the performance issues in one fell swoop.
But what do I know eh?
11.01.07
Yeah, I've not been able to get anything related to Turbine Dump since I've taken this new job. I've got to get moved! I leave for work at 6AM and get home around 8PM. That's far too long of a day to expect to get anything of substance done away from work.
Man, I'm tired!
Man, I'm tired!
07.01.07
After some discussion over at Turbobricks about options in turbochargers, it's about time to make a decision. Any more pondering will border on Barney'ism.
The objective of the car is simply to be a killer street car with a broad power band of useable power. Even as it it now, I've accomplished that objective. The car has mad grunt on the bottom end and absolutely steller roll on performance. The only downside is that I've not met my power goal which at first was 250hp, but has been moved to 300hp. 285 at the very least.
Anyway, with the turbo I have now (TD04H-15G), it's not likely I'll make that point with the next iteration of the engine. Well, it's possible but I doubt it will happen.
The next iteration includes a ported, polished, and grooved 530 head with a B cam. The B cam has a good deal more lift and moves the power spread a little further upwards in the rev range bu not too much that it will jepordize the our objective of streetability. Additionally, others have reached more then 300hp with that cam, though admittedly using larger turbos then I.
Now there is also a worst case scenario to the next iteration of the motor/turbo setup. As it is now, my car is smoking and I'm pretty positive that's from the turbo. However, there is an outside chance that it's actually the block and oil getting past the rings at idle. While I certainly hope that's not the case, it slows things down by at least a month. The good side of such an outcome is that it'll force me into a 90+ block, thereby removing any lingering reservations about the K block.
So back to the turbo stuff: The most likely choices, instead of those crazy expensive GT2871R's I was pondering before, are the Mitsu 18G-5H, Evo III Big 16g, or a T3/T4 with a .48 hotside and a 60-1 cold side. All of these turbos are very will priced and small enough on the hot side to make very little in the way of lag from my 2.3. The only drawback of the Mitus listed above is the need of a $40 dollar T3/DSM adapter. It's actually more of a risk in terms of the potential fitment issues caused. I guess that's the kind of thing we won't know for sure until we "go there".
There is of course one more option that also has had some discussion. That is using an 18g or 19t wheel in the 15g compressor housing. However, when you consider that I have to pay for the work AND the turbo CHRA still needs to be rebuilt, it may end up being more expensive then just buying a new turbo.
Anyway, all of these are very well priced options compared to the ball bearing GT family.
The objective of the car is simply to be a killer street car with a broad power band of useable power. Even as it it now, I've accomplished that objective. The car has mad grunt on the bottom end and absolutely steller roll on performance. The only downside is that I've not met my power goal which at first was 250hp, but has been moved to 300hp. 285 at the very least.
Anyway, with the turbo I have now (TD04H-15G), it's not likely I'll make that point with the next iteration of the engine. Well, it's possible but I doubt it will happen.
The next iteration includes a ported, polished, and grooved 530 head with a B cam. The B cam has a good deal more lift and moves the power spread a little further upwards in the rev range bu not too much that it will jepordize the our objective of streetability. Additionally, others have reached more then 300hp with that cam, though admittedly using larger turbos then I.
Now there is also a worst case scenario to the next iteration of the motor/turbo setup. As it is now, my car is smoking and I'm pretty positive that's from the turbo. However, there is an outside chance that it's actually the block and oil getting past the rings at idle. While I certainly hope that's not the case, it slows things down by at least a month. The good side of such an outcome is that it'll force me into a 90+ block, thereby removing any lingering reservations about the K block.
So back to the turbo stuff: The most likely choices, instead of those crazy expensive GT2871R's I was pondering before, are the Mitsu 18G-5H, Evo III Big 16g, or a T3/T4 with a .48 hotside and a 60-1 cold side. All of these turbos are very will priced and small enough on the hot side to make very little in the way of lag from my 2.3. The only drawback of the Mitus listed above is the need of a $40 dollar T3/DSM adapter. It's actually more of a risk in terms of the potential fitment issues caused. I guess that's the kind of thing we won't know for sure until we "go there".
There is of course one more option that also has had some discussion. That is using an 18g or 19t wheel in the 15g compressor housing. However, when you consider that I have to pay for the work AND the turbo CHRA still needs to be rebuilt, it may end up being more expensive then just buying a new turbo.
Anyway, all of these are very well priced options compared to the ball bearing GT family.
22.12.06
Well, figuratively speaking at least. When I get back home tonight I will prolly go out for a spin even though I shouldn't.
The problem relates back to my hybrid 15g install. It's a 13c hotside/15g coldside. When putting it together I definitely noticed some in/out play at the back (exhaust) of the CHRA. As a result, it's letting oil past. While idling it will let lightly mist the area with blue smoke. That's not good! If I let shut the car off then restart it a couple of moments later, it will smoke fer shizzle and leave a thicker cloud behind.
So while it is fun, I really need to get it up in the air to deal with a host of things as well as change out the turbo. I really want a GT2871R or 3071RWG. Those are mad turbos and would go nicely with this motor. Especially the 2871 for low end and mid range responsiveness.
Oh well..... A guy can dream right?
The problem relates back to my hybrid 15g install. It's a 13c hotside/15g coldside. When putting it together I definitely noticed some in/out play at the back (exhaust) of the CHRA. As a result, it's letting oil past. While idling it will let lightly mist the area with blue smoke. That's not good! If I let shut the car off then restart it a couple of moments later, it will smoke fer shizzle and leave a thicker cloud behind.
So while it is fun, I really need to get it up in the air to deal with a host of things as well as change out the turbo. I really want a GT2871R or 3071RWG. Those are mad turbos and would go nicely with this motor. Especially the 2871 for low end and mid range responsiveness.
Oh well..... A guy can dream right?
12.12.06
Well, it looks like I finally go back to work after looking for some time. I'll be employed with a company (start up) called Pay Per Post. It looks like it's going to be a blast. I'll actually be coding in Ruby and Ruby on Rails. I spent the weekend hackin' as a matter of fact. After going dark with a copy of "Ruby In A Nutshell", Google, and plenty of coffee, I hacked out a small command line app that stores phone numbers and emails for people.
Anyway, and for those that don't know, the language is a lot like Python. The differences?
* The interpreter doesn't squawk about indentation
* ABSOLUTELY EVERYTHING IS AN OBJECT
* Has some funky iteration stuff that is very concise and perl'esque as a result
* A lot of funky operators in the place of words. ('<' for 'extends' as an example)
I'm also not happy with what I had to jump through to generate a multi-dim hash. I may yet find a
better way to create dim hashes, but it certainly wasn't as easy as it was in PHP.
But otherwise, it's a rockin' language and I think I'm going to like using it.
Anyway, and for those that don't know, the language is a lot like Python. The differences?
* The interpreter doesn't squawk about indentation
* ABSOLUTELY EVERYTHING IS AN OBJECT
* Has some funky iteration stuff that is very concise and perl'esque as a result
* A lot of funky operators in the place of words. ('<' for 'extends' as an example)
I'm also not happy with what I had to jump through to generate a multi-dim hash. I may yet find a
better way to create dim hashes, but it certainly wasn't as easy as it was in PHP.
But otherwise, it's a rockin' language and I think I'm going to like using it.
23.11.06
This weekend would be a perfect weekend to cut the springs on my car. I just don't want to do anything. It's Thanksgiving and when you get right down to it, I don't even need the car. Some of it has to do with the fact that I don't feel I have the proper tools so I'm just not comfortable digging into it.
OTOH, I think I'll hook up that RCA cable for my sub and make a rock sheild for my oil cooler. Those are things that can't possibly leave me without a car between here and Tuesday when I go out of town for an interview in Boca Raton. I think I'll change the water pump too since there is some racket that seems to be coming from that area.
OTOH, I think I'll hook up that RCA cable for my sub and make a rock sheild for my oil cooler. Those are things that can't possibly leave me without a car between here and Tuesday when I go out of town for an interview in Boca Raton. I think I'll change the water pump too since there is some racket that seems to be coming from that area.
I'm really hip on the idea of deleting this blog completely. Not sure what to do. It was getting pounded from trackback and comment spam. I just got tired of it but it's time to do something about it (I've said that twice before (I've said that twice before). The comments table was over 60 megabytes. I finally just dumped it to a file, gzip'd it, and truncated the table.
b2 is the worst code in the world.
b2 is the worst code in the world.
19.01.05
It's about time! Google finally does something concerning comment spam. [General] -
BDKR - bdkr@highsidecafe.com @ 14:30:54
Jason posted a link to something on Google (dated today as a matter of fact) that should make bloggers all over Dodge happy. Basically it'a an attribute (rel="nofollow") that should be added to any hyperlinks generated by the comment (or whatever) functionality of ones blog software. It's obvious then that the Google Spider will not follow these links and any gain from the comment in the first place becomes nill.
Now this doesn't stop the comments from coming in. But once spammers realize there is no gain, they will most likely loose interest in placing the comment in the first place. Which is the goal of this move by Google in the first place. So in the short term, it's still sheilds up for those who haven't updated their blog package (or home grown package) to add the nofollow attribute.
Now this doesn't stop the comments from coming in. But once spammers realize there is no gain, they will most likely loose interest in placing the comment in the first place. Which is the goal of this move by Google in the first place. So in the short term, it's still sheilds up for those who haven't updated their blog package (or home grown package) to add the nofollow attribute.
16.01.05
Tu sei per me...iaria che respiro
il mio cibo, cioche osservo
magico sei tu! e sento che
Sono inamorata di emozioni
smisurate e infinite ormai!
vivi, nei miei pensieri.
a velveteen equation
magico, questo eun grande amore
magico, questo eun grande amore
il mio cibo, cioche osservo
magico sei tu! e sento che
Sono inamorata di emozioni
smisurate e infinite ormai!
vivi, nei miei pensieri.
a velveteen equation
magico, questo eun grande amore
magico, questo eun grande amore
15.01.05
Harry, who in my opinion is an extremely thoughtful chap has been up to some interesting things as of late. With the title of his latest blog being Procedural PHP leads to slower apps, he was sure to get some interesting comments and sideways glances.
That aside, someone brought up the point that there is a huge issue with tribalism in the prog com (Programming Community) and noted that it also exists with relation to programming methodologies. OOP vs Procedural.
Click here is another good link concerning advocacy and how not to go too far, shall we say?
What is a lot more interesting than the actual topic, is that this is a superb example of tribalism. (Going meta is always fun!) You identify "your own group" with the positive figures, those that know how to do it right and conveniently blend out all the "failures", since those don't belong to the group yet - they're doing wrong. (Also see "no true scotsman fallacy") Whereas the opposite group is always identified by their worst members - low-lifes, thugs, rapists, spaghetti-coders. Please forgive my trolling. Have a read at this timeless article: http://www.perl.com/pub/a/2000/12/advocacy.html It's only superficially about programming languages, it applies in much the same way to programming methodologies. Posted by: Tribalist Jan 14th, 2005 @ 8:56 AM MSTThe story he is talking about is here, just in case you can't get to it from the blockquote above. It really is a war that doesn't need to be fought. Anyway, many know my feelings on the whole OOP vs Procedural thing and I hope it can be seen that I feel that both have their place. The only thing that is unfortunate to me is the idea that "Procedural must be bad". Unfortunately, we see a lot of this. Additionally, there is the extension of the an 00 centric view onto data management. If you want more info on this topic, look into the writings of Fabian Pascal and C.J.Date. A bit of a tough chew, but worth it!
Click here is another good link concerning advocacy and how not to go too far, shall we say?
01.01.05
Well, although I've come a long ways towards being able to defend against the stuff, I'm still getting attacked. It seems as if there has been some campaign launched for the new year. Between my two blogs I've seen at least 200 (I haven't really counted) attempts. Most of them for the old Texas Hold 'Em garbage. Just take a look at my log file. Starting today, I've also taken to storing IP addresses as well. Beyond a certain number of occurrences for an IP I may consider redirecting the bot elsewhere. Maybe the Better Business Bureau? Don't know.
Anyway, I've also written a script that will parse the attempts list and run whois requests for the URL's in the posts. I just don't know what I would do with this information. Any ideas?
Cheers
Anyway, I've also written a script that will parse the attempts list and run whois requests for the URL's in the posts. I just don't know what I would do with this information. Any ideas?
Cheers
21.12.04
I am hoping this site will have one. I am at work now, but I will be trying to contact the person here when I get home from work. I'm just posting in the event someone looking for info on this topic happens across this post.
Cheers...
http://weblogtoolscollection.com/archives/2004/12/04/dynamic-blacklist-ready-to-be-used/
Cheers...
http://weblogtoolscollection.com/archives/2004/12/04/dynamic-blacklist-ready-to-be-used/
18.12.04
While messing around in the database trying to figure out how I may be able to deal with the spam, it occured to me that a good deal of it could get gone with a script that could come through nightly and delete offending comments. Much of this is based on the use of b2, but this can apply to any of the systems methinks. It's based on the fact that we may not have to match against an exact string. That said, using the SQL locate() string function, I could search for the existence of these terms in the author or url fields in the comments table. I could most likely do the same for the actual comments themselves, but I fear it may delete legitimate posts from those that are merely commenting on the term. I will have to think about this aspect a little more.
Anyway, I now have a cron job that will clean the database nightly. It works against a list of search terms and urls that I have saved in .txt files. I would love to get this kind of information from a universal source. That way, as opposed to updating it myself to try to keep pace with the spammers, I could automatically update the files on a periodic basis. While there may be issues with such a system (read comments here), I feel they can be worked out.
Now I don't think this is a final solution in any way shape or form. But I think it can help. There are a number of other things that can be done and some of those I have plans to do in the next couple of days. This same list of authors and urls can be checked up front. If any of the terms are found to exist in a comment submission, fail the procedure! Plain and simple as that.
In closing, before I started looking at the stuff the day before yesterday, I had over 3500 comments (the vast majority of it junk) in the comments table. I'm now down to a 10th of that number. There are still a number of terms that I need to add to the terms files yet, but what is there plus what I did by hand before writing the script has done well.
Lastly, b2 has some serious performance issues that got progressively worse as the dataset increases. As can be guessed, this is a databse issue and the fact that there are not enough indices. Since so much of what's going on here revolves around the b2posts and b2comments tables that it's a must to index more than just the primary keys here. Creating and index for the date operation for both tables is very good idea. It also seems that the b2comments tabls was ignored even more than b2posts. Creating indices for comment_post_ID has a huge effect on performance.
If anyone is interested, the file is in the tools directory. The terms files are there as well.
Cheers,
BDKR
Anyway, I now have a cron job that will clean the database nightly. It works against a list of search terms and urls that I have saved in .txt files. I would love to get this kind of information from a universal source. That way, as opposed to updating it myself to try to keep pace with the spammers, I could automatically update the files on a periodic basis. While there may be issues with such a system (read comments here), I feel they can be worked out.
Now I don't think this is a final solution in any way shape or form. But I think it can help. There are a number of other things that can be done and some of those I have plans to do in the next couple of days. This same list of authors and urls can be checked up front. If any of the terms are found to exist in a comment submission, fail the procedure! Plain and simple as that.
In closing, before I started looking at the stuff the day before yesterday, I had over 3500 comments (the vast majority of it junk) in the comments table. I'm now down to a 10th of that number. There are still a number of terms that I need to add to the terms files yet, but what is there plus what I did by hand before writing the script has done well.
Lastly, b2 has some serious performance issues that got progressively worse as the dataset increases. As can be guessed, this is a databse issue and the fact that there are not enough indices. Since so much of what's going on here revolves around the b2posts and b2comments tables that it's a must to index more than just the primary keys here. Creating and index for the date operation for both tables is very good idea. It also seems that the b2comments tabls was ignored even more than b2posts. Creating indices for comment_post_ID has a huge effect on performance.
If anyone is interested, the file is in the tools directory. The terms files are there as well.
Cheers,
BDKR
16.12.04
The comment spam got out of hand a long time ago. I've just been ignoring it. However, when I realized that I am helping pornographers and off shore gambling houses make money and suck up space in my DB at the same time, I needed to get a grip on the situation. I worked a good deal at looking at the code for b2 last night to see what could be done. Got fed up and went to be without making any real headway.
Interestingly enough, there is a very recent blog entry on this topic by Harry Fuecks talking about a couple of things along the same lines. A number of solutions are talked about and there is a large number of links off to various places with some good examples.
As for solutions, some of you are prolly aware that I implemented a turing test on the comments section, but as it turns out, the spam is still coming. So that means ...
some Joe is getting payed to key these in, right along with the turing example.
it's an automated script of some sort.
there is another avenue for adding comments in the api that I am not aware of.
For the longest time I thought it was the last option above, but what I found last night suggests that there is instead an army of desperate skank gnomes keying these things in by hand. Now the approach taken by MT-Blacklist is a bit more involved than just a simple turing test, but to the best of my knowledge (correct me if I'm wrong) stops short of requiring a log in. Unfortunately, it requires the use of MT. I'm using the orginal b2, which is the largest collection of worst practices on the net. I have yet to look at it in any detail, but there is still an issue as pointed out in the blog.
So what then? What about this idea?
He even went so far as to suggest a method for dealing with spammers that register with the system. I like this idea best.
Anyway, in the short term, I still have the turing working on part of the site (as it turns out, there are multiple ave's for entering comments) and I may still try a bit of inline filtering using the code (or my version of it) provided by one of the commenters. Kind of a 'bad word' list approach.
My "die" comment will most likely be a little more diplomatic than this.
Anyway, a single blog comment registration system would be awesome in my opinion. I think it's worth a try. But oh well. I may end up just taking the site off the air for a bit while I clean all these foul birds from out of the trees.
Interestingly enough, there is a very recent blog entry on this topic by Harry Fuecks talking about a couple of things along the same lines. A number of solutions are talked about and there is a large number of links off to various places with some good examples.
As for solutions, some of you are prolly aware that I implemented a turing test on the comments section, but as it turns out, the spam is still coming. So that means ...
For the longest time I thought it was the last option above, but what I found last night suggests that there is instead an army of desperate skank gnomes keying these things in by hand. Now the approach taken by MT-Blacklist is a bit more involved than just a simple turing test, but to the best of my knowledge (correct me if I'm wrong) stops short of requiring a log in. Unfortunately, it requires the use of MT. I'm using the orginal b2, which is the largest collection of worst practices on the net. I have yet to look at it in any detail, but there is still an issue as pointed out in the blog.
Distributing blacklists still has the problem that is is subject to junk being added to it, but this time from an infinite number of locations rather than at a single source.
Ideally, borrowing some concepts from BIND, you could setup a distributed black network for any arbitrary list (eg. email spam, blog spam, bad servers, bad ip's, bad words, etc) that works between trusted servers. If there were enough trusted servers, people download a copy of the list from a local server. Then it would just be a matter of resyncing the lists between the servers at some defined interval.
The only other solution that has been mentioned here that has any real benefit is pre registration of users. If you guys really want annonymous posts, then no matter what you do, your asking for trouble. That's why mail relays and annoymous ftp servers are bad now, because the whole point of them is you have no idea who is giving you the data, good or bad.
Pre-registration and authentication seems to me that it would cut out something like 90% of any spam you are likely to get providing you make the registration process difficult enough that scripts like snoopy cant just create users on the fly to circumvent it.
Posted by: mrsmiley from realityedge.com.au Dec 15th, 2004 @ 2:47 PM MST
So what then? What about this idea?
Since everyone's main concern is 'popular' software, (including the spammer-script author who wouldn't want to write a script for single-use) why not just setup a internet-wide registration, kind of like 'yahoo groups' or that one message board (cant remember it's name) did. You register ONCE for the 'blogging system', then, when you come across a blog that utilizes this system, you're cookied in, or you have to login/pass with your system-wide login/pass. This could be done on 3rd party (vendor) SSL popup w/ api's back&forth if password security is a concern. Basically, then, if someone is reported enough times for spamming, they get added to the blacklist (any form of blacklist will do, to be debated later) and then their posts can be 'hidden', and they can be prevented from posting AT THE DISCRETION of the blog maintainer. how's that for power to the people? You could even setup thresholds (# or reports from ACTUAL blog maintainers), and rate the blog maintainers by some other metric so as to prevent a malicious user from signing up for a blog & trying to blacklists everyone who posts on their blog.
anyway... a game of monopoly anyone?
~synace~ http://www.synace.com
Posted by: synace Dec 15th, 2004 @ 12:10 PM MST
He even went so far as to suggest a method for dealing with spammers that register with the system. I like this idea best.
Anyway, in the short term, I still have the turing working on part of the site (as it turns out, there are multiple ave's for entering comments) and I may still try a bit of inline filtering using the code (or my version of it) provided by one of the commenters. Kind of a 'bad word' list approach.
| <? if ( 'sunny@moonlightshadow.us' == $email || preg_match('/poker|casino|loans|gambling/i', $url) || preg_match('/phentermine|consolidation|play-texas/i', $url) || preg_match('/debt consolidation/i', $author) || preg_match('/viagra/i', $comment) || preg_match('/ownthis\.com/i', $url) || preg_match('/texas-hold/i', $url)    { die( __('I do not accept comments from you. Buzz off.') ); } ?> |
My "die" comment will most likely be a little more diplomatic than this.
Anyway, a single blog comment registration system would be awesome in my opinion. I think it's worth a try. But oh well. I may end up just taking the site off the air for a bit while I clean all these foul birds from out of the trees.
Yep, you guessed it. Another endless meeting that got nowhere. A boss that has a grand long range vision but doesn't want to take the immediate steps to help himself reach those goals. I really need my own gig!
15.12.04
Something occured to me that wasn't pleasent. By allowing comment spammers to continue to hit my site with that garbage, I'm helping them out. That said, I'm going to do something about it. I added some code to the comment popup function that forces one to enter a code that's randomly generated, but I failed to realize that there is another avenue for posting to this site. I need to plug that hole.
I think the reason I haven't done it sooner is because the b2 code is so bloody messy. It's the most convoluted broken bowl of logic I've ever seen. There is logic in it and it does work, but understanding it is allways a bit tough.
OTOH, one of the major sites I'm responsible for at work has some of the same issues. I've been able to clean it up and even create a template for the administration portion of the site. I've come a long ways towards understanding what's going on with it. Now if I could do it with that site, why would I not be able to do the same with b2?
Oh well, time to get to the spaghetti bowl...
I think the reason I haven't done it sooner is because the b2 code is so bloody messy. It's the most convoluted broken bowl of logic I've ever seen. There is logic in it and it does work, but understanding it is allways a bit tough.
OTOH, one of the major sites I'm responsible for at work has some of the same issues. I've been able to clean it up and even create a template for the administration portion of the site. I've come a long ways towards understanding what's going on with it. Now if I could do it with that site, why would I not be able to do the same with b2?
Oh well, time to get to the spaghetti bowl...
14.11.04
While checking a post at my favorite message board (Turbo Bricks), I saw a link posted to an entry at the North American Subaru Impreza Owners Club. In short, this kid with board name of 'Panache' was blitzin a neighborhood in his WRX and lost control, shooting through a house, it's fence, and the neighbors fence. The crazy thing is that he gets online joking about it with a fake story about how it happened.
Well, there are a number of interesting things here.
This is the effect and power of Social Software. Anyway, even after 13 days, the site still seems to be getting pounded. Either that or it was allways slow. But one way or another, I'm now very curious as to how well their systems as a whole scaled.
Well, there are a number of interesting things here.
- NASIOC got /.'ed with out ever having a post on /.!
- This episode has shown that the fabric of cultures on the internet are more tightly woven then we may have been used to thinking.
- There are so many links in and out that a search for 'panache' and 'wrx' at Google lands this thread at the top of the list.
This is the effect and power of Social Software. Anyway, even after 13 days, the site still seems to be getting pounded. Either that or it was allways slow. But one way or another, I'm now very curious as to how well their systems as a whole scaled.