Quick WP upgrading with WPCLI

This is the easiest way to upgrade WordPress. You’ll execute these commands on the server itself.

Requirements

  • ssh access to your server
  • wp-cli command installed (instructions for installing wp-cli at http://wp-cli.org/)

Install/Upgrade WP CLI

  • wp-cli should be upgraded each time a WordPress installation is upgraded.
wp cli update

Upgrade WP

Prep work

Change into the WP directory

cd /path/to/wordpress/installation/

Make a list of active plugins

wp plugin list --status=active --format=csv --fields=name | tail -n +2 > ../active-plugins.txt

Update all plugins

wp plugin update --all

Deactivate all of the plugins

wp plugin deactivate --all

Upgrade WordPress

wp core update

Reactivate all of the previously active plugins.

cat ../active-plugins.txt | xargs wp plugin activate

Check the site in various browsers (make sure cache has been cleared).

OSSEC, Suhosin, and WordPress

I had a problem show up on some of our servers. Visiting sites would work fine, but as soon as you log in to a WordPress site, then your IP was blocked at the firewall level. It took a bit of hunting around the OSSEC logs to figure out the cause, and then finally tipped off to a local rule to combat the blockage. I outline the process of figuring out what was wrong, and how to fix it below.

DENIED!

gareth-davies-logo3So initially, this was quite confusing. All of a sudden people would have their IP blocked. I checked the different sites, and seemed to have no problem. Then when I logged in to the back end, BAM, blocked as well. We have Shorewall running, so doing:

shorewall show dynamic

Showed all of the IPs that Shorewall had blocked. This could also be done by using iptables

iptables -nL --line-numbers

Sure enough, my IP had been blocked. I unblocked my IP with:

shorewall allow ip.ad.dr.es

Or could also do

iptables -D INPUT 2

where INPUT is the position or section of the firewall chain, and “2” is the line number containing my IP address.

Then I checked with other web applications on that server. Where they also causing an issue? I logged in to an Omeka install. No problems.

FOUND IT!

ossec_logoOK. I know OSSEC is to blame some how. It’s an awesome HIDS (Host Intrusion Detection Software) that actively responds to issues on the server based on scanning through the system logs and various rules.

OSSEC keeps itself chroot’ed to /var/ossec/, so all of the ossec logs are located in /var/ossec/logs/.

I first looked in the /var/ossec/logs/active-responses.log. Sure enough, a couple of lines like this show my IP being completely blocked to the server.

Fri Jun 14 06:50:47 EDT 2013 /var/ossec/active-response/bin/host-deny.sh add - XXX.XX.XX.XX 1371207047.5913585 20101
Fri Jun 14 06:50:47 EDT 2013 /var/ossec/active-response/bin/firewall-drop.sh add - XXX.XX.XX.XX 1371207047.5913585 20101

So, there we are. OSSEC blocking the IP for some reason. Now why is it blocking the IP?

Taking a look in the /var/ossec/logs/alerts/alerts.log file to see why it thinks it needs to block the IP…

** Alert 1371206381.5698606: - ids,
2013 Jun 14 06:39:41 (server1) 127.0.0.1->/var/log/messages
Rule: 20101 (level 6) -> 'IDS event.'
Src IP: XXX.XX.XX.XX
Jun 14 06:39:40 server1 suhosin[18563]: ALERT - script tried to increase memory_limit to 268435456 bytes which is above the allowed value (attacker 'XXX.XX.XX.XX', file '/var/html/wp-admin/admin.php', line 109)

There were other lines in there with my IP, but nothing would/should have caused blocking, like a WordPress login event, or an SSH login event. Also, the error above is categorized as an IDS event with level 6, which by default OSSEC rules means it gets blocked.

HOW TO FIX IT!

As a quick fix, I changed the “suhosin.memory_limit” option in /etc/php.d/suhosin.ini to 256M, and the “memory_limit” in /etc/php.ini to 256M so that no error would be generated.

Now came the hard part of finding out how to fix it for real. OSSEC is a pretty big beast to tackle, so I turned to my friendly web search engine to help out.

To fix the issue, I would need to write a decoder or new rule to ignore the suhosin rule causing the problem. OSSEC has descent documentation to get you started, but fortunately the blog linked above had the solution already. https://www.atomicorp.com/forum/viewtopic.php?f=3&t=5612

From user ‘faris’ in the forum linked above:

Add the following lines the the /var/ossec/etc/rules.d/local_rules.xml file.

<group name="local,ids,">
  <!-- First Time Suhosin event rule -->
  <rule id="101006" level="14">
    <if_sid>20100</if_sid>
    <decoded_as>suhosin</decoded_as>
    <description>First Time Suhosin Event</description>
  </rule>
  <!-- Generic Suhosin event rule -->
  <rule id="101007" level="12">
    <if_sid>20101</if_sid>
    <decoded_as>suhosin</decoded_as>
    <description>Suhosin Event</description>
  </rule>
  <!-- Specific Suhosin event rule -->
  <rule id="101008" level="5">
    <if_sid>101006,101007</if_sid>
    <match>script tried to increase memory</match>
    <description>Suhosin Memory Increase Event</description>
  </rule>
</group>

What these new rules do is change the level of the default rules that were tagged/decoded as being suhosin errors. In the first rule, if the default error is 20100, and is decoded (or tagged, or matches) as suhosin, then set the level to 14 instead of the default 8.

The second rule detects if the default error 20101 is decoded as coming from suhosin and sets the level to 12 instead of the default 6.

The third new rule looks at any error tagged as suhosin and if the error has the matching text in it, then it sets the error level to 5 (below the limit for firing an active response).

So, just add that group of rules to the local_rules.xml file and restart the OSSEC service. BA-DA-BING! no more blocking the IP when logging in to WordPress.

Convert WP to static HTML – part 2

This is a followup to this previous post.

So I’ve been converting some more blogs to static html files, and this time around things seemed to be so different, that I made up a new how to. Here are the steps that I’ve been using to convert blogs using the default Kubric theme.

  1. Update the permalink structure for the site so that it uses the year, month, day, postname structure.
    UPDATE `database`.`prefix_options` SET `option_value` = ‘/%year%/%monthnum%/%day%/%postname%/’ WHERE `prefix_options`.`option_name` = ‘permalink_structure’ LIMIT 1 ;
  2. Make sure the blog does not block search engines. If the blog is set to block them, wget can only download the index.html file. And this took me a while to figure out. So, for the sake of search engines, if wget only downloads the index.html file or wget recursive gets only index.html file, then remember to check your robots.txt or similar settings. Either edit in the admin section (under Settings->Privacy) or via SQL.
    UPDATE `database`.`prefix_options` SET `option_value` = '1' WHERE `prefix_options`.`option_name` = 'blog_public' LIMIT 1 ;
  3. Add the .htaccess file if not already there, where
    /path/to/wordpress/blog/

    starts at the URL root, not the absolute file path. So http://sitename.com/path/to/wordpress/blog/ would have the .htaccess file below in the ‘blog’ directory.

    RewriteEngine On
    RewriteBase /path/to/wordpress/blog/
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule . /path/to/wordpress/blog/index.php [L]
  4. Get rid of the meta links through the sidebar widget in the admin, or delete the appropriate lines from the theme files (for default Kubric theme edit comments.php, sidebar.php, single.php, footer.php), or see the last step. Delete the code that puts in the search, comments, trackback, rss, and anything in the footer you want out.
  5. When all is good, run wget to grab the files.
    wget --mirror -P blog-static -nH -np -p -k -E --cut-dirs=5 http://sitename.com/blog/
  6. Rename the blog directory. mv blog blog-old
  7. Rename the static directory to be live. mv blog-static blog
  8. Copy the images directory from the old theme to the appropriate static directory.
    cp -r blog-old/wordpress/wp-content/themes/default/images/ blog/wordpress/wp-content/themes/default/
  9. Alternative to get rid of unwanted links, etc. Use the find command to find all html files, then use perl to delete the lines. Don’t forget to escape forward slashes in the search field. Unfortunately, this method requires you to do it for every line of code you want to delete. It’s much better to delete the lines out of the theme files. The code below has an unnecessary space in the opening H3 tag so it will render properly.
    find . -name \*.html | xargs perl -ni -e 'print unless /&lt; h3&gt;Leave a Reply&lt; \/h3&gt;/'

    Also, if you want to just search and replace instead of remove, this handy find and perl one-liner will find and replace text in all html files.

    find . -name *.html | xargs perl -p -i'' -e "s/search text here/replace text there/"

    The above would search for all the “search text here” phrases in all html files, and replace it with “replace text here”. You can obviously substitute whatever you want in those to places. If you have a ‘/’ (forward slash) character, it will need to be escaped with a ‘\’ (back slash) character. Perl uses the regular regular expression syntax, so look that up if you need help formulating a search and replace structure.

Major Update to Multiple SVN WordPress Installs

It’s been a while, but I thought it important to post an update to the wpupdate program I wrote to upgrade a whole mess of WordPress installs at one time. I took a cue from the program officially sponsored by WP, but think mine is much, much better. 🙂

Here are some of the features:

  1. Specify a file with a list of svn WP installs, or update the current directory, or specify the directory to update on the command line
  2. Use command flags or options. You can specify the program to update or switch, use tags or branches.
  3. Automatically saves a copy of the svn update to a file so your terminal is not overflowing with text, but does output any conflicts that arise.
  4. Automatically saves a backup copy of the wp-contents directory (just in case the update or switch screws something up).
  5. Automatically saves a copy of the database, backing up only the tables used by the WP install (based off of the wp-config supplied table prefix).
  6. Restores permissions to the original owner and group.

Anyhow, check out the new page devoted soley to this application: SVN WordPress Updater

License: Anyone is free to use this program however they want, as long as they give me due attribution. Also if you update or modify the program in any way, I need to know about it. That’s what free and open software is all about. Any updates should benefit us all.

THAT podcast

Check out THAT podcast (THAT = The Humanities And Technology). It’s a new video pod cast put on by a couple of co-workers at CHNM. They interview someone in the technical field about software that helps those of us in the humanities.

The first episode includes an interview with Matt Mullenweg, creator of WordPress (the software running this site!) and shows you how to install and configure ScholarPress (a plug-in to WordPress written by Jeremy Boggs).

It’s great stuff, check it out!

Converting WordPress to static html

UPDATE: Check out the new post on a better way to do this here: Convert WP to Static HTML Part 2. Or see the page devoted to the script here: Make WordPress Static.

Usually people are wanting to convert their static html pages to some dynamic content management system. I’ve run into the issue of needing to go the other way.

A few professors at GMU love to use WordPress for their classes. It’s a really great way to get more student participation and involve some of those who aren’t so talkative in class.

But these blogs are usually only needed for one semester, and then just sit there. This can be a security risk if they are not kept up to date, and is cumbersome when trying to update many of them (one professor had over 30 blogs!).

Sometimes the content should still be viewable, but the need for a whole cms type back-end no longer exists. Sometimes the professor would just like a copy of the pages for their own future research or whatever.

So, I figured out a way to convert a dynamic WordPress site into static html pages.

Here are the basic steps I used:

  1. Change the permalink structure in the WordPress admin section. Alternatively, directly in the database change wp_options.permalink_structure.option_value to “/%postname%.html”.
    [code lang=”SQL”]
    UPDATE `database`.`prefix_options` SET `option_value` = ‘/%year%/%monthnum%/%day%/%postname%/’ WHERE `prefix_options`.`option_name` = ‘permalink_structure’ LIMIT 1 ;
    [/code] 

    UPDATE (2.12.08): Reading a post from Christopher Price (who linked to this post) about WP permalinks, I’m thinking using this structure (/archives/%post_id%.html) might afford the best results. I often found a page that displayed the raw HTML instead of being rendered. This just might fix that issue.

    UPDATE (3.11.08): I did some more dynamic to static conversions today, and found out the best permalink structure to use is just the post name. No extra categories and such. So the best structure to use would be this (/%postname%.html). The benefit is that the every page is unique with a descriptive name for the url (albeit sometime very long), and there are not as many subdirectory issues that arise.

    UPDATE (7.17.09): This time around, I have found that the following seems to work best for permalink: /%year%/%monthnum%/%day%/%postname%/ And cleaned up the SQL statement.

  2. Add the .htaccess to /path/to/wp/ if not already there (where /path/to/wp/ is from http://somedomain.com/path/to/wp/ ). If there already is a .htaccess file and it is set to have permalinks, then you can probably leave it as it is.
    RewriteEngine On
    RewriteBase /path/to/wp/
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteRule . /path/to/wp/index.php [L]
  3. Use wget to copy all of the files as static html files.
    [code lang=”bash”]wget –mirror –wait=2 -P blogname-static -nH -np -p -k -E –cut-dirs=3 http://sitename.com/path/to/blog/[/code]
    *** Change –cut-dirs to the appropriate number associated with how many directories are after the domain name. The trailing slash plays a part too. ****
    UPDATE (03.11.08): I found that the –cut-dirs doesn’t really do anything this time around.
    UPDATE (7.17.09): This time around, I find the following to work best, even the –cut-dirs. 

    wget --mirror -P wpsite-static --cut-dirs=3 -nH -p -k -E https://site.com/path/to/wp/

    This has the bonus of making the directory for you, thus negating the make directory step. Make sure to use two dashes and not an em dash.

  4. Copy the contents of wp-content to save uploaded files, themes, etc. This way copies a lot of unnecessary php files, which could be potentially dangerous, but is really easy if you’re just converting to archive. To remove the security threat, just pick and choose the files you need.
    [code lang=”bash”]cp -r /path/to/wp/wp-content/* /path/to/static/wp-content/[/code]
  5. Sometimes the files are created with folders in the archives folder. To fix this run the following three commands in the archive folder to fix that up. To get rid of the feed file in all of the directories:
    [code]rm -f */feed [/code]
    To delete all of the now empty direcotries:
    [code]find . -type d -exec rmdir ‘{}’ \;[/code]
    To rename the files ###.1 to ###
    [code]rename .1 ” `find . -type f -name “*.1″`[/code] That’s two single quotes after the first ‘.1’
  6. UPDATE (03.11.08): I have found that the old ‘rename‘ command [rename .1 ” *.1]only works on the current directory. If you want to do a recursive renaming you have to use the ‘find‘ command. The above code has changed to reflect this.
    UPDATE (7.14.09): When the rename with find doesn’t work, it’s probably because the post has comments, so there is a folder with the same name as the post’s filename. In this case, just move the file (with the .1 extension) into the folder of the same name, but change the name of the file to index.html

  7. move to wp folder. make a backup of database: [code lang=”bash”]mysqldump -u [userfromwp-config.php] -p –opt databasename > databasename.sql[/code]
    UPDATE (03.11.08): I found I needed to backup just a few tables from a database that contained many copies of wordpress. To do this more easily, I used a little script I wrote earlier to dump tables with a common prefix. This could also work if you just put in the full name of only the tables you wanted to backup.
  8. move one directory above wp install. make tar backup of old wordpress folder: [code lang=”bash”]tar -cf wordpress.tar wordpress/[/code]
  9. rename the old wordpress folder [code]mv wordpress wordpress-old[/code]
  10. move the static copy into place [code]mv static/wordpress/ wordpress/[/code]
  11. test out the site. If it’s totally broke, just delete the wordpress directory and restore the original from the tar file.
  12. remove the tar file and wordpress-old directory as needed.

Adding the calendar in WordPress

This is the info…
http://codex.wordpress.org/Template_Tags/get_calendar

but to save you all from having to go to a different page, I shall explain it here.

You simply need to add a line of PHP code to the template. The code is:

< ?php get_calendar() ? >

The steps:
1. Go to the admin section of WordPress.
2. Presentation
3. Theme Editor
4. Sidebar
5. Find where this part of the code is:

< ?php wp _list_pages('title_li=< h2>Pages< /h2>' ); ? >
	< li>< h2>Archives< /h2>
		< ul>
		< ?php wp _get_archives('type=monthly'); ? >
		< /ul>
	< /li>

6. and add the calendar code before the wp_list_pages junk.
7. update the page and BANG! You’ve got a calendar.

*Be aware, I had to add some spaces in the above code so that it would show up properly.