Spam Filters and Dopey IT guys!!

[bitingmidge]

Obviously if you are into IT and clever enough to be reading this you are not being referred to in the header!!

So! Over the last few months at work outgoing emails have been disappearing at random, and incoming just not getting in!

In spite of many denials by the IT chaps who get paid by the hour not to fix these sorts of things the problem continued.

One of the companies concerned was a major (capital M) law firm, and I had sent three identical messages with large attachments (each in two parts=six total) over a two week period, only the last ones arrived.

I was less than happy with the performance of the said firm, as I had no "return message" and amost thought they may not be telling the truth.

After yet another nil response from them, I sent a rather sarcy sort of email which resulted in an immediate response from the senior partner of the firm who had personally emailed me.

He advised that he had his IT gurus looking at the problem, because they had lost a number of important documents in recent times.

Off I go. Both off my nut, and off in search of why.

To shorten a very long story, that particular firm has a standard header on its emails incorporating its phone number : 3**6 666*.

Our "new" spam filter has a standard default which decided the numbers 666 were included in a nasty bit of spam, and blocked messages FROM the individual adress after 5 receipts. Others in the firm were blocked at random, and our outgoing messages to blocked individuals were also intercepted and shredded!!

I've read a bit about this sort of thing but never experienced it, but had to stand over the IT chap with a big hammer to suggest how to track down the cause (actually to even convince him it was at our end!), and white list the address!

No one has yet offered a satisfactory answer for the seeming "randomness" of the intercept. Are spam guards like customs officials, only opening every fifth package for inspection or what???

Please cast some light on this for me!!~

Cheers,

P :D :D :D

[Cliff Rogers]

As an "IT" guy of sorts, I know that SPAM filters are like 2 bob watches.... more trouble than they are worth. :(

The ISP that hosts our domain name has some very weird anti-spam rules so I have a Pigpond account set up I can use for anyody who is having trouble getting e-mail to & from us via our normal e-mail. The Pigpond account works that well that I'm considering shifting my domain name hosting to them instead. :rolleyes:

[outback]

666 - The devil made me do it. At least that's what Hanse reckoned. :D

[Sturdee]

666 is a high rank with the Hongkong Triads.

May be they have been trying extortion tactics on them by way of email. :D :D :D


Peter.

[knucklehead]

Midge, we had a lass that just could not get her email to come through. Every one just patted her on the head and treated her like a bimbo that must be doing something wrong.

Eventualy it was traced back to the spam filter. Her name was Heather, the filter decided that HE AT HER constituted an offensive email and deleted it.
Therfore any email with her name on it was deleted.

[q9]

The problem is most IT people seem to think that inappropriate content is their problem to solve. I was recently working for a large gvmt dept here in qld, and I can't believe the rubbish they went on with...utterly mind boggling.

While you shouldn't have it a free for all, nor should it be locked down so tight that you lose more than you gain. But IT people in general love stuff like this, because they like to have the "power" that comes with the administering of the software, and because they "know" what is best for you.

It really is an attitude thing, and the reason a lot of IT people should be fired. And before anyone decides to lynch me for being anti-IT, I should mention I get to put "Bach IT" at the end of my name...

[ubeaut]

I have taken Spam filtering into my own hands with Spam Bully which downloads all of the spam (and we get heaps around 1600+ a week (243 this morning since last night)) into a file in Outlook. I can then do a quick scan manually to see if there are any that have been grabbed as spam by mistake at the moment we are getting about 2 or 3 a week. These are then classified as good email and won't be grabbed again. This has so far been the most reliable filter I have found especially for someone with multiple domains and email addresses.

Does take a little bit of time to go through but it is now around 99.8% accurate the best I have ever been able to get before this was around 98% and that's far from satisfactory as far as I'm concerned.

Cheers - Neil :)

PS Spam Bully is pretty discerning and probably wouldn't have picked up the Heather and 666 bits but if it had, it could have been corrected the first time and would never have happened again.

[arose62]

'Heather' reminds me of an early contract I did on a VAX/VMS system.

A new guy called Tissa Tilekeratne (gee, I still remember after, what, 20-ish years), saw me redirect some screen output into a file. He asked about the trick, I gave him a demo, and away he went.

Later on he complained that it wasn't working, so I wandered over to his desk where he showed me what he was doing. He'd used his initials "tt" as the file name, but in VMS that's a reserved word for the terminal, so everything he sent to "tt" was just disappearing.

I would have changed my name.

Cheers,
Andrew

[kiwigeo]

Dont get me started on IT help desks.....

Last job I came off my laptop hard disk blew up (not surprising as it was a Dell). Try as I might to convince the IT help person that the job should be assigned priority1 (=cant do my job without the problem fixed) the job stayed on priority 3 (=minor problem) for the 10 days it took to get it fixed.......NO computer for 10 days .

[q9]

That is surprisingly long for dell service. I've never had to wait more than a day to get a tech on site/part to arrive for anything.

[Stuart]

No one has yet offered a satisfactory answer for the seeming "randomness" of the intercept. Are spam guards like customs officials, only opening every fifth package for inspection or what???

Please cast some light on this for me!!
Only thing I can think of, is the "randomness" is caused because the spam filter is a learning filter- ie as it gets potential hits on spam confirmed, it learns from this and applys more intellient filtering the next time. It will oscillate around this point, getting more precise in its filtering with each iteration.

In general, for (non-corporate) solution to spam, I cannot recommend Mozilla Thunderbird highly enough. It is free (www.mozilla.org), written by the same people who created Netscape (that is before it was sold off).

It's built-in spam filtering is a learning filter, and so long you are diligent in telling Thunderbird what is, and isn't spam it gets smarter and smarter.

I personally get up to 200 spams/week (care off Monash Uni's email address list being compromised). However, after using Thunderbird for the last few months, and being diligent with the spam filtering, I now will get ?1? spam email / month in my inbox. All the others are deleted by Thunderbird automatically after 7 days. (In the meantime, it stores them in a 'junk' folder that I can check to make sure legitimate ones haven't been caught).



IT people wouldn't have such an attitude about "knowing what is good for you" if more people took some responsibility for the tools they use.

[Barry_White]

IT people wouldn't have such an attitude about "knowing what is good for you" if more people took some responsibility for the tools they use. Gee Stuart

That made us all smart a bit.

But must agree about Mozilla Thunderbird I never get any spam but it transfers a few e-zines I get to the Trash folder that have some words in them it thinks are spam.

[Stuart]

Sorry Bazza, and all others -(quite a) bit of work bias coming through.

IT Instructor, and Helpdesk supervisor for the past 4 1/2 years (or should I say survivor). After a while, you can get to the point where 90% of the problems are caused by 10% of the people, and 99% of them could save themselves a lot of grievance if they just spent some time learning how to use the tools provided, instead of whining.

And before any personally directed negative comments about typical helpdesk attitudes- I was given the Vice-chancellors award last year for the improvements I made to service support from our IT helpdesk. Got me a new Jet bandsaw, and a new digital video camera out of it! Ok, tooting my own trumpet - sorry again.

Bazza, regards to Thunderbird, have you been unchecking any 'spam' that got caught by mistake? I assume that you have been, but if not, Thunderbird will learn from this.

[kiwigeo]

That is surprisingly long for dell service. I've never had to wait more than a day to get a tech on site/part to arrive for anything.

Dell werent doing the service.....was doing a job for the worlds biggest oil company...all the IT support is handled out of the USA. Even getting a password reset takes 3-4 days with the same company.

[kiwigeo]

Sorry Bazza, and all others -(quite a) bit of work bias coming through.

IT Instructor, and Helpdesk supervisor for the past 4 1/2 years (or should I say survivor). After a while, you can get to the point where 90% of the problems are caused by 10% of the people, and 99% of them could save themselves a lot of grievance if they just spent some time learning how to use the tools provided, instead of whining.

Stu...apologies on my part as well in case my post gave the impression all IT help desks operate the same as the one I was dealing with. Ive worked for clients with really good IT operations but unfortunately most of my clients are really big oilers and one thing Ive found is the bigger the company is the more they function like small governments.

Cheers Martin

[Stuart]

Hey, no probs, just a small dummy spit on my part (I need that icon!)

I ***** about our central IT support as much as any (probably louder)

[q9]

Funnily enough, the best place I ever worked for was a uni also. We provided same day service for nearly everything, and 99% of jobs had a 24 hour turnaround.

In the business world, which I expected to be roughly the same, it was much, much worse. IT departments seem to exist as a seperate service provider, rather than an intergral part of the business. A lot of the IT guys I have worked with seemed to have no concept of the idea that one person not being able to log on WAS a big deal. No thought of productivity of the BUSINESS, only what is convenient for them to deal with. They like to impose bizare 'security' policies that make no sense, or are completely inappropriate, and in every single case where that has been done, all it has done is increase the workload of the IT helpdesk, and impede on the legitimate work of the end users.

Anyhow, SPAM is a worldwide problem, outside the control of the end user, and even the IT dept. So how does blaming the end user help? I would suggest that if you are losing a large portion of legitimate email to your spam filter, then it is inappropriately configured.

I am afraid I don't have much time for support people blaming the end user for their woes. I agree that some people are their own worst enemy, but it is up to the support professional to ensure that they are appropriately managed. This often does not occur, and generally it is because it is difficult, time consuming and (wrongly) thought to be a waste of resources.

Anyhow with luck, I will be back at uni studying something a little more challenging and worthwhile next year ;)

[Stuart]

Anyhow, SPAM is a worldwide problem, outside the control of the end user, and even the IT dept. So how does blaming the end user help? I would suggest that if you are losing a large portion of legitimate email to your spam filter, then it is inappropriately configured.

I am afraid I don't have much time for support people blaming the end user for their woes. I agree that some people are their own worst enemy, but it is up to the support professional to ensure that they are appropriately managed. This often does not occur, and generally it is because it is difficult, time consuming and (wrongly) thought to be a waste of resources.
Re; Spam, I'll pay all that. We have a good spam package, that gives control to the end user to manage it as they want. The package (spam assassin) rates each email as to it's likelyhood of being spam. It is up to the end user to decide what to do with it. We had a major spam even about a month ago (as did the rest of the world - care-of the Germans) and some users had 100s of emails because they hadn't paid any attention to the numerous notifications given as to how to set up spam filtering. They kept blaming the IT dept, when we are not in a position to enforce what they decide to do. I also offered training courses, which a massive 2 people out of the 1000 or so in my faculty chose to attend (the courses are free, so no excuse there).

Being a Uni, we have a different approach to the corporate world. Freedom of activity is given, as it is an academic environment, and therefore we do not want to be a compromising factor for academic research. It does mean however, that the end-user must take more responsibility for the tools they use, and the computer is just another tool. If they choose not to make use of the helpdesk support and training support offered before they have a problem, then demand assistance when something goes wrong (such as loosing all their data because they have not given any consideration to backup), then yes, the most common issue the helpdesk has to deal with is PEBKAC (problem exists between keyboard and chair). We proactively deal with any other problems.

[pau1]

Being a Uni, we have a different approach to the corporate world. Freedom of activity is given, as it is an academic environment.
I'd have to disagree with you there stuart. I work as a systems administrator at a uni. Among other things I also run our mail server and associated spam scanning. If anything our current policies are tighter and more secure than the "corporate" environment. The only people with administrative access to desktop boxes are the systems administrators. We don't have virus problems, we don't have malware and illegal programs installed, we don't have to re-install boxes at the drop of a hat. All that and staff still get all their work done and still have the "freedom" to do so. We have run like this for the last 6 years and staff appreciate the value of stable and always usable desktop machines. btw I work for an IT faculty.

We also use spamassassin and only tag the mail as spam/non-spam. No mail is lost and it is up to the mail client to decide what to do with it. The active non-delivery of mail by filters on the server is a bad policy and only leads to disatisfied end users. You can stop 100 spam and the end users won't care, but stop one legitimate email and you will hear about it.

[DavidG]

Surely the way to stop spam is to remove the concept of free e-mail.

If the user paid a small charge for every e-mail sent, over a free limit per account and a small joining fee to create an account, then spam would cost the spammer more than it was worth but legit users would hardly be effected.

At the moment you can create as many free accoounts as you like and mail what you like from each. Too easy.

Sender pays.

[Cliff Rogers]

...If the user paid a small charge for every e-mail sent... Sender pays.
Too simplistic, most spammers are not using their own account to send the spam anyway.

It also won't stop them... do you think the junk mail stuffed in your snail mail letter box gets there for free? :)

Anyway, as I have said before in the topic about telemarketers, you'll live a lot longer if you don't let it bother you. :D

[pau1]

Surely the way to stop spam is to remove the concept of free e-mail.

If the user paid a small charge for every e-mail sent, over a free limit per account and a small joining fee to create an account, then spam would cost the spammer more than it was worth but legit users would hardly be effected.

At the moment you can create as many free accoounts as you like and mail what you like from each. Too easy.

Sender pays.
That was the solution proposed by Bill "The Internet? We are not interested in it" Gates

Anyway half the spam sent out is from compromised virus infected machines that set up their own smtp servers and spam the world. Either that or they are comming from some dodgy country that doesn't quite subscribe to the same "laws" as the rest of the planet and they just do whatever they want. Large quantities of Spam aren't being sent by people who go and set up "free" webmail accounts, the origins are often much more sophisticated.

[q9]

Emailing is very simple to do through code. It only takes a few lines and you have created a basic email servlet type of thing.

A lot of spam would be reduced if antivirus and firewall software had to be shipped with new computers. In fact, I wrote something like that to the Dept of Information and the Arts once upon a time. You'll notice that their "anti-spam" legislation has done exactly what I predicted it would do at the time - nothing. I suggested that it would be better to legislate that all new computers have AV and firewall software installed.

Of course, people will cry that they are paying for something they don't want. But your ISP is charging per MB - as it costs them per MB - there would have to be a saving in there somewhere, even if only 10% fewer computers were being compromised.

[DavidG]

But.... If you had to pay then you would be much more careful with your system.
You would use a firewall as it could save you money. You would take every precaution. You would scream at the pollies to get a solution....

Because there is no immediately visible cost, resources are not allocated to prevent misuse. What does it matter to you if your system does send a few hundred thousand emails. A little extra traffic hidden in the rest.

You would react pretty quick if you had to pay for a few hundred thousand emails. Very quick.....

[Gingermick]

My solution requires some police and legislative work. What we need to do once we catch one spammer is to punish them severely. This would involve the forcible placement of anti-virus and anti-spam compact discs (and maybe ADSL modems) into the personal orifice from whence brown stuff emanates. Then the use of chastity belts to keep the discs there.

[Cliff Rogers]

But.... If you had to pay then you would be much more careful with your system.....
....You would react pretty quick if you had to pay ....
Yes, & this is from somebody who puts a picture in their signiture? :rolleyes:

What about the poor buga's that have to pay by the Mb for the download??? :(
They are paying to read this thread.... you won't stop people just by making them pay.

[DavidG]

Cliff
It is actually a link to ubeaut. It should be in your cache after the first time.

I would like to get it reduced in size so have to ask neil.

This was in response to issue that people using this board did not know about the ubeaut products so I thought we could advertise them.

[Cliff Rogers]

Still, you get the point, it's advertising that you have to pay for yourself.... making e-mail 'user pays' won't stop SPAM. That pic in your signiture is still 3Kb, it adds up.

It doesn't stop the junk mailers that have to pay for all the trees they cut down to print the sheyet that they stuff through my mailbox every single day of the week so it won't stop junk e-mail either.

[DavidG]

Cliff
Yeh. and your avatar is 7k.

Nothing will stop them but it might slow them down.
The stuff in the snail letter box is targeted and they hope to make a profit.
Spam is blindly sent by the million.

Price them out and fine them out. Then fit a desk top pc up their rear end, plugged in of course.

[Cliff Rogers]

Cliff
Yeh. and your avatar is 7k.
.& your point is? :(

My point is that you won't stop them by trying to make them pay, you won't stop them, they will do it anyway.


....The stuff in the snail letter box is targeted ......Rubbish.... who said I wanted a Manicure, perfume, Barby dolls & stainless steel saucepans:rolleyes: & what idoit thinks that they can sell me a crap computer when I work in a computer business? :cool: Targeted my ass, it's the same thing as SPAM & they pay for it to be delivered.

If you want trageted, that's what Spyware is all about & who likes that? :mad:

Ignore it & chuck it in the bin, you'll live longer. :)

[pau1]

But.... If you had to pay then you would be much more careful with your system.
You would use a firewall as it could save you money. You would take every precaution. You would scream at the pollies to get a solution....

Because there is no immediately visible cost, resources are not allocated to prevent misuse. What does it matter to you if your system does send a few hundred thousand emails. A little extra traffic hidden in the rest.

You would react pretty quick if you had to pay for a few hundred thousand emails. Very quick.....
Who pays whom? Do I pay you? Do I pay Bill Gates, my isp, the government, or the derro on the corner? The internet isn't run by one company or government or isp. If it was, there probably wouldn't be a spam problem.....because the internet wouldn't exist. Even if you did make people pay, whats to stop them writing a new "email" system that uses different protocols and different ports etc If it was free and it worked I'm sure it would catch on pretty quick.

Payment isn't the sollution. Trust me, I'm payed to stop spam for a living and read more articles and research on the problem than I care to admit. If there was a quick fix then it would have been implimented. The basic problem is that the smtp protocol is insecure because when it was designed (based on FTP) they had no idea that it would be one of the largest comunication tools of the new millenium, so security wasn't built in. There are technical ways of fixing the problem, but it's "too hard" for any one country/company to do. It will come, but you will probably have to wait until e-mail becomes unusable due to spam:mad:

[q9]

... Targeted my ass, it's the same thing as SPAM & they pay for it to be delivered...

I actually worked for a firm that spent a lot of money on marketing...in fact that's nearly all they do. Mail outs are VERY targeted. Maybe not to you personally, but definately geographically, based on a number of factors. Your proximity to certain stores, the economic make up of the suburb in which you live, average house prices in your neighbourhood, and a bunch of other stuff like if a previous customer lives in your street. Mailouts (where I was at least) only achieve something like a 1% success rate, but that 1% usually generated several million dollars of extra revenue, for a cost of only maybe $50,000.

There, now that's nicely off topic, isn't it :D

[Stuart]

http://www-128.ibm.com/developerworks/library/lol/spamato/spam-c07.jpg

[DavidG]

Why do I feel like I am talking to the IT department again.

At this point I agree to disagree.

[q9]

Problem with user pays is simply this:

The software to create a mail server is free. You only need a very basic hardware setup to install a basic mailserver. All you need after that is just a simple connection to the internet. So it becomes easy to set up for a couple of days then move on, or perhaps setup in another country where no one will bother you about such things.

So how do you ensure the user pays when they can do everything for free and move about so easily? Essentially, you can't.

[spbookie]

Problem with user pays is simply this:

The software to create a mail server is free. You only need a very basic hardware setup to install a basic mailserver. All you need after that is just a simple connection to the internet. So it becomes easy to set up for a couple of days then move on, or perhaps setup in another country where no one will bother you about such things.

So how do you ensure the user pays when they can do everything for free and move about so easily? Essentially, you can't.

These days the majority of spam is sent via hacked machines, so called "bot nets" you can hire 1000 computers for a day for about $30US. So now who pays? Another issue with charging for email is that you're are alreadly paying for email in your connection cost so how do you feel when someone tries to slug you for extra? I used to run three mailing lists and I would send approx 3000 emails a day, should I be charged more for providing a community service?

It is not easy. I think a good start to solving the problem is when an ISP detects that a user's computer has been compromised they shut off all access until it is fixed. Having a firewall and anti-virus running as a condition of connection would help heaps too.

Internet connection, being a cooperative, is a privlige not a right.

Charles
Who is behind two seperate firewalls, with local AV and FW and I'm wearing a ubueat foily cap!

[kiwigeo]

Nice pic Stu.....the computer looks like an IBM portable.

[kiwigeo]

Charles
Who is behind two seperate firewalls, with local AV and FW and I'm wearing a ubueat foily cap!

Charles, if youre wearing a ubeaut foily hat then you dont need the AV or FW!!

[Daddles]

Nice pic Stu.....the computer looks like an IBM portable.

In which case, it's coming OUT of the computer and INTO the tin :eek:

Richard

[Barry_White]

When I changed from Telstra ISDN to Telstra 2 Way satelite a condition of the connection was that you were running a firewall and anti-virus software. Coupled with using Mozilla Thunderbird I do not get any spam.

[Stuart]

In which case, it's coming OUT of the computer and INTO the tin :eek:

Richard
Isn't this where canned spam comes from?