Saturday, November 26, 2016

Everything's Great Until It's Not - Hacked Websites, What to Do About Them and How to Prevent a Hacking

Earlier this year, I was hired to repair a hacked website for a Los Angeles-based entertainment company.  Ever since then, I have been knocking the concept for this post around in my head.  With the City of Dunwoody's website hacking, the subject has come to the front burner and I decided it was time to get this one posted.

Security is Relative

There is no such thing as a 100%-hack-safe computer on the Internet.  Whenever I see RFPs requesting "100% uptime" in their website, I take that as a red flag that someone doesn't understand computer security.  Even NORAD and NASA were hacked by teenagers.    A developer or network admin can make hacking extremely difficult - so much so that a hacker decides the site isn't worth their time and moves on.  But if a hacker wants to get into a computer system, they will eventually find a way in.  Website owners and hosting admins have to be vigilant and plan for recovery from attacks before they happen.

All Hacks Are Not the Same

Hacked websites will appear and behave differently based on the intent of the hacker.  In Dunwoody's case, somebody wanted to make a point and make sure the entire planet knew who they were and what was on their mind.  My entertainment company hack was more insidious:  the site looked the same but the administrator could not log on.  The hacker had made numerous changes in the back end that not only allowed clandestine access rights but also made it difficult to track down and remove the rogue code.

You Got Hacked!  What Now?

The name of the game is to eliminate any code or other elements that were placed by hackers and make sure your actual website code is intact.  How much work is involved is going to depend on what the hacker did once they got access to your site or server account and in some cases, how much stuff you had stored there in the first place.

This may or may not be a DIY job, depending on how comfortable you are with the files and database that drive your website.  When in doubt, consult a professional.

If you have a simple website and your webhost provides a routine backup option (or "snapshot", as its called by some) you can simply restore an older version of the website to replace the hacked version.  This requires some planning ahead and I'll go into those plans further down.

But what if you didn't plan ahead and there are no backups to restore?  That's when you have to take the step-by-step approach.

The Most Complete Hack Job I've Ever Seen

My entertainment site hack client was an example of a very deliberate, stealthy, even surgical hack that left the public face of the site intact but prevented legit administrators from accessing the dashboard, allowed different admins access, and hid the hacked content inside of normal files.  Here's what I found.

  1. The hacked site was in WordPress.  Which had not been updated in years.  The site also used a commercial theme purchased from a developer.  Which also had updates available and also had not been updated in years.  The host was bare-bones GoDaddy.  Without a site backup option. Problem #1 was identified.  The owner provided access to the GoDaddy account and the files could be read through a browser.
  2. There were a ton of directories with obsolete code from earlier versions of the website.  Inside were recently-added PHP code files with benign names like "security.php" and "file.php".  But they weren't benign - they were part of the admin control hack.  The tip-off was the date when they were added:  when all of the files are 2,3,4 years old and you have a new file that was added last month, that's your culprit.   The delete key was my friend.
  3. Similar benign-sounding-but-not files were found in directories that housed javascript files that drive the interactive and animated elements of the website.  When you have a directory full of files that end in ".js" and there's a recently added file ending in ".php", that's a file that doesn't belong there.  Delete key again.
  4. Same process was repeated in the "uploads" file where images and the like are kept.  When you see a bunch of ".jpg" and ".gif" files and a new one out of nowhere ending in ".php", that's the file you delete.
  5. Database edits:  the hackers created an additional table in the WP database to override the admin commands.  They also added new users with administrator roles.  Delete the extra table and delete any and all users that were not currently authorized on the site.  This is where a professional database admin comes in handy and we leave the DIY Zone.
  6. The last piece of the puzzle was the worst.  The hackers had not only done ALL of the above, but they had inserted lines of code into actual WordPress files, either at the very beginning, or at the very end.  Now if you're a developer who is used to looking at the individual files that make up WordPress and reading the code, you can tell when something is added that doesn't belong there.  There is NO way a casual user would have caught this.  Even if they managed to spot and delete all of the above hacks, the result would have been the site crashing altogether.  I reviewed all of the files in the core content management system and removed code that wasn't kosher.
BINGO!  The administrators could log in.  Software got updated, including the outdated theme and my customer was updating the website again.  No further hacks.

The above cleanup process took about 12 hours and I charge my hourly rate for that kind of work.  Can that situation be avoided?  Oh hell yes.  And I highly prefer it to fixing a site that had been entered by somebody using a server in the Czech Republic.

Six Steps To Avoid the Highway to Hacker Hell


  1. This is no longer a joke, gang.  If you are one of the many millions on Planet Earth using some kind of content management system (CMS) whether open-source or commercially available, apply all of your updates when you are aware of them.  The purpose of these updates is to close those little loopholes where hackers get through.  That's the core software.  Plus the little add-ons - "plugins", "modules", "extensions", whatever they're called.  And your themes.  Or "templates", whatever they're called too.  Most of these can be done automatically through the administrator's dashboard.  Or even from a VAN DOWN BY THE RIVER!!!

    Contact the CMS' manufacturer's website (or tech support if you're using a proprietary software like Sitefinity).  Don't put it off.  If you can't work it into your schedule, hire a pro to review it regularly.  That's cheaper than recovery.

  2. BACKUP, BACKUP, BACKUP  If your webhost has an automatic backup function, turn it on and use it.  If they don't, back up your files manually and find a new webhost that offers this service.  Backups refer to both the files where your data is displayed, and the database where the data is stored.  Most competent web hosts have help files and/or tech support that will walk you through this.  When in doubt - hire a professional and have it done for you regularly.  Monthly, at the very least for the typical non-government small business.

  3. Remove any obsolete files or directories from your web host.  If you're not using old plugins or other code, delete it from your server.  Save it to a local drive for posterity if you want but it's just a trouble magnet on the Internet.  Follow your CMS' instructions for uninstalling and removing components for your website.  Old directories with code no longer in use need to go.  If you're not sure how to do this ... wait for it....  hire a professional.

    WordPress users:  even if you uninstall and delete a plugin properly, the data from that plugin may still be left behind in the database.  That can still be a vulnerability, albeit a rare one.  There are plugins available that can clean out "orphan" data from a database but a database admin can manually go through and remove anything no longer in use.

  4. Remove any users who are no longer working on the site.  I believe this may have been part of what led to the major hacking I cleaned up.  There were a number of users still listed on the site who edited content and plugins from time to time, but were no longer involved.  They were still there, same passwords in place.  A disgruntled former employee can do a lot of damage.  But even if there are no bad feelings, what if that former employee gets hacked and someone gets a hold of their login credentials?  The result is the same.  Just delete the user and their access.  If they return, give them a new login.

  5. Rotate your passwords.  I know, passwords are a PITA to remember and use.  Changing them on a regular basis can prevent hacking and other fraud.  Don't forget to make it something that a hacker won't be able to find using a standard algorithm of searching for common words.  Many CMS and hosting providers require a minimum security level, including a mix of numbers, upper and lower case letters, and punctuation.  Anyone who still uses "password" or "123456" needs to be slapped.  As well as kept away from computers!

  6. Reevaluate your hosting provider periodically.  Not all hosts are created equally.  The uber-cheapo hosts are great for hosting vacation pictures by Aunt Suzie from Syracuse.  E-commerce needs something more secure, and that will require an investment.  Do some research as your web presence expands to make sure your website's foundation is secure and has the power to run your applications without hanging or crashing.  Government agencies and municipalities - don't skimp on this, no matter what the citizenry says.  When I consult for any government office or agency, I have a very short list of providers I will work with.

    Amazon web hosting is the gold standard for high-volume performance and security.  To give you an idea of what they are capable of, Amazon hosts, and a number of federal agencies, including the CDC here in Atlanta.  In the private sector, they host NetFlix.  But hosting like that costs big bucks - there is no price menu at Amazon, all contracts are negotiated especially the high visibility ones.  Most enterprises do not need this level (or cost) of power and security but the point is, this isn't a place to go cheap.

It's all about paying regular attention to the bones of your website.  Either set aside the time or hire someone who will.  The following does not have to be you, if you apply some regular due diligence.

Wednesday, November 23, 2016

Happy Thanksgiving 2016

As usual, the sun doesn't set on an entrepreneur so between baking pie, roasting turkey and making sure the kids get out and run off some energy, I'll be upgrading software for my e-commerce clients in advance of Cyber Monday.

I hope all reading this can find some peace this week, through Christmas, Hanukkah, and the New Year.

Usually I post a brief excerpt from Nathaniel Philbrick's Mayflower regarding how building a community takes hard work and compromise.  Not going to cut it this year.  Here's the link if you'd like to review.

After the kids are settled in for the night Pat and I will split a bottle of pinot noir and a viewing of Places in the Heart.

Not a typical Thanksgiving movie.  Or any holiday movie for that matter.

Places represents Sally Field's 2nd Oscar win.  The movie is set in Great Depression-era Waxahachie, Texas (yes, that's a real town, just outside of DFW).  Won't spoil the plot if you'd like to see it but my takeaway is that family and community can grow anywhere - whether you want it to or not - in places you can't imagine.  Forces outside that family can modify it, but not destroy it if you don't let them.  (Warning:  given the date and setting there is some blatant racial violence in a few scenes.  Parents, judge carefully and be ready for an intense conversation if you allow your children to watch.)

Also a movie I recommend for Sisterhood nights in Phi Mu or any sorority chapter gathering.

We'll be home for turkey day, so it's OK to stop by and pass judgement on my cooking.  :-)

Monday, November 14, 2016

DHA Speakers Video Recap: GLASS, Dunwoody Senior Baseball, school contribution requests

In case you missed it...

Dunwoody Homeowners Association's monthly meeting played host to several groups last night.  The following videos were streamed live on my Facebook feed.

All of these are publicly available.  Please view and share and continue the conversation in your neighborhoods.

DHA will meet again in December.  Come and join in live and in person next time.  Until then, we'll see you at Light Up Dunwoody on November 20 at the Cheek-Spruill Farmhouse!

GLASS (Georgians for Local Area School Systems) legislative update

Dunwoody Senior Baseball presents their case for changing the IGA and the new baseball field location.

School funding requests from Kingsley and Vanderlyn.

Tuesday, November 8, 2016

Monsters From the Id - Social Media and 21st Century Politics

Forbidden Planet, released in 1956, was a groundbreaking movie in a range of aspects, only a few of which were in special effects.

The "planet" was previously home to a technologically "superior" race - the Krell.  During the course of the movie, we learn this civilization had created an infrastructure that could manufacture anything and deliver it to anyone with a thought.  It sounds like a great idea when Robby the Robot was creating jewels and clothing for Altaira or whiskey for the spaceship's drunken cook.

But the technology couldn't differentiate between a conscious wish and a subconscious nightmare.  So when the Krell slept, and the "Id" came alive, their technology delivered their greatest horrors.  No one was spared - not the most intelligent, not even the creators of the technology.

It is now 2016 and it is time to cut the bullshit daydreaming that our voting communities can be "united" after venting themselves on the Internet during election season.

The US Civil War was 150 years ago, and even though the USA is legally one country, the cultural prejudices and hostilities between North and South persist to this day.

Neither Trump nor Clinton will "unify" the US, regardless of who wins today.  It's unreasonable to call your opponent's supporters a "basket of deplorables" and expect them to fall in line with your inauguration.  It is equally unreasonable to make unfiltered knee-jerk smart-ass comments about your personal nemeses day after day for months and think all will be forgotten on November 9.

It's not just national politics.

In my little burg, the greatest advocates of "preservation" have torn elements of our community to pieces all in the name of keeping physical relics alive.  Our elected officials have turned to petty dictators in an effort to mold our city into their personal living room.  And ran afoul of the state government within 6 months of our last municipal election.  But that isn't new.  For years one representative or another has advocated for one group's lifestyle in this city to the detriment of others.  All in the name of "building community".  I have a flash for you, Barney - you don't create a community by turning elements of it against each other over residential lifestyle differences.

Then there's the nuclear arsenal of the carpool set:  school redistricting.  We have our playdates, we have our chance meetings on the playground.  We have our scouts, and youth groups and sports teams.  Then the idea of rearranging our school populations rolls in like a live grenade and the monster devours everyone again.  Think I'm exaggerating?  Read for yourself and decide.  You can figure out the personalities with a few minutes of research.

With all of the blog posts and social media clips and screenshots - nothing will be forgotten. No matter how many photo ops are taken or how many charity projects supported after the fact, everyone will remember getting attacked and hurt.   Everyone will remember who was on what side, and what attack they made.  Forgiveness is another topic altogether and I'm not holding out too much hope for that either.

The Krell planet eventually found peace - when the population disappeared and there was no one left to dream up a monster.

The best any elected official at any level can hope for in 2016 is a cessation of hostilities.  An equilibrium between groups whose POVs are irreconcilable.  It takes superhuman objectivity and a committed refusal to engage in combat online.  That's a very short list of capable people.  No one can achieve it perfectly.  But "unity"?  Forget it.  It's gone.  Mutual coexistence while minimizing overt hostility is the best you can shoot for.

I can't watch the returns tonight.  Likewise when keeping up with local politics, I do it alone, in a closed office.  Best when social media is turned off.

Migrating databases, debugging source code and managing household paperwork has never looked so good.  It's peaceful and it puts the Id to sleep.