Here at Cadence Labs, whenever we on-board a new client, the first thing we do is audit their Magento source code. This helps us provide our clients with security recommendations and accurate estimates for future projects or upgrades.
If you are interested in having your Magento website audited by one of our senior developers, please take a look at our Magento Audit Services.
We highly recommend taking the time to perform a thorough, methodical, and informative audit before diving into a new codebase. Unless the client is using the vanilla rwd theme with no extensions, chances are there are several pieces of important custom functionality hovering beneath the surface. If you’re trying to upgrade Magento, you will absolutely want to audit the codebase to ensure you’re not losing any functionality in the process.
Step 1: Get a development copy of the site
This should go without saying, but before doing any work on the new codebase, make sure to create an independent development copy that’s hosted on a different server from production. Copying the site is beyond the scope of this tutorial, but there are several easy-to-follow guides on google.
Step 2: The Tools You’ll Use
I want to cover some useful tools for performing audits:
- n98-magerun: https://github.com/netz98/n98-magerun – Describing itself as the “swiss army knife for Magento developers” n98 is great tool to more quickly interface with your Magento system.
- Magento Project Mess Detector: https://github.com/AOEpeople/mpmd – we’ll use this tool heavily, it will perform a methodical diff on the codebase, comparing it to a “clean” / vanilla Magento install. This is an extension for n98
- XDebug: https://xdebug.org/ – this is essential to doing php development. XDebug provides enhanced error reporting, and the ability to run full code traces / profiles.
- Varien Profiler – this is built into Magento, however here is a great guide by Inchoo on using it: http://inchoo.net/magento/keeping-your-magento-fit-with-built-in-profiler/
- Google Page Insights: https://developers.google.com/speed/pagespeed/insights/ – we’ll use this tool at the end to do some basic benchmarking.
Step 3: Audit Areas
Next we’ll cover the areas of the codebase that we plan to audit, and their significance:
Core Code Changes
Magento encapsulates its native code in the app/code/core and lib/ folders. Magento provides a number of ways to override / alter the functionality of code in these directories without changing them directly. Any changes in the core code are bad news – for a few reasons. First, it’s not apparent to other developers what areas of the core have been changed. Speaking for myself, I always assume the core code hasn’t been touched (hence why I run an audit). Secondly, if you upgrade Magento, all of these changes will be overwritten / lost forever. This is why it’s critical that you run an audit before you upgrade.
Local Copies / Overrides
One of the ways that Magento allows us to modify core code (without corrupting our installation) is to create a local copy. This involves moving a file from app/code/core/Mage into app/code/local/Mage. The way Magento’s include paths work and Magento always looks for a file in the local codepool prior to searching in the core pool. Although this approach is better than modifying core code, it’s less ideal than creating a module to rewrite the class in question. When running our audit we’ll want to be aware of what files (if any) are app/code/local/Mage.
Magento Security Patches
Magento consistently releases security patches which update the core code to fix vulnerabilities. We’ll want to scan the website to see if any of these patches are not applied as part of our audit.
Theme Best Practices
Magento provides a number of ways to modify the base theme (or a commercial theme) to only change what’s absolutely necessary. Mainly, each theme has a local.xml that gets loaded last. This file is intended for modifications to your theme. Often we see Magento sites where other developers have copied catalog.xml, customer.xml, sales.xml, etc into the current theme just to change one or two lines of code. The optimal way to update this code is via using local.xml (See some cool tricks here: http://magebase.com/magento-tutorials/5-useful-tricks-for-your-magento-local-xml/ )
Log Files / Log Cleaning
We’ll want to examine Magento’s log files and log database tables and answer two questions: 1) Are there any critical errors in the exception.log or system.log? and 2) How big are the log files and the log tables? If they’r enormous, we’ll want to find a way to keep their size under control.
Developing an enterprise level website without version control is extremely dangerous. When you’re auditing a site, look to see if the previous developers were using a VCS – if so, you’ll want to save a copy of the repository. I won’t be including this as a full section below, it’s more of a note to those of you who made it this far!
We’ll check to make sure best practices are being followed relating to css/js compression, static asset expiration dates, and image optimizations.
Extensions / Custom Functionality
Every Magento site (usually) has a number of extensions or custom modules developed. We’ll want to get a birds eye view of what extensions are in use.
When 2 or more extensions attempt to rewrite the same Magento class, you have a conflict. By default, Magento “resolves” this conflict by ignoring one of the rewrites! This means your site may be missing some functionality. We’ll get a list of all these conflicts and then find a way to resolve them.
Step 4: Audit Core Code
You’ll want to follow the instructions to download both mpmd and n98. For the purposes of this demo, I’m pretending I have a clean Magento install located in ~/clean_magento and a customized install in ~/dirty_magento
You’ll want to chdir into the dirty_magento directory and run:
n98-magerun.phar mpmd:corehacks /path/to/clean_magento /path/to/report.html
This will create a report.html file for you at the given location. You can view this in your browser and should see something that looks like:
As you can see, this gives us a lot of useful information. It will tell us what core files are missing, changed, new, etc. Under the “different file content” we even have the option to view an individual diff for each changed file! Based on the number of core code changes, you can make some initial judgements around the health of the codebase.
Step 5: Audit Local Copies
Once again we’ll be using n98 / mpmd – this time we’ll run:
n98-magerun.phar mpmd:codepooloverrides /path/to/report.html
This will create another report that will show us codepool changes:
Step 6: Audit Security Patches
There is a really easy tool for auditing security patches! Enter magereport.com – by visiting this site, we can get an overview of what vulnerabilities exist on the site (including some that are not related to a patch but are general best practices [e.g., admin report url, downloader protected etc]).
Finally, we can check for all applied patches by viewing: app/etc/applied.patches.list – this should be the complete log of all patches applied to the site. However, this file *might* have been lost, thus magereport.com is your best bet for getting a true picture of what vulnerabilities exist.
Step 7: Audit Theme Best Practices
This one is tricky. It’s very subjective, and you need to have some experience working with custom Magento themes. First things first, discover what theme the site is currently using by visiting System -> Configuration -> Design and looking for the current theme / package. You’ll then want to find the corresponding theme in app/design/frontend/package/theme.
Once there, answer the below questions:
- Does the theme extend the base theme or another commercial theme?
- (If yes to #1) – Is the theme using local.xml or selective layout.xml overrides to modify the page design? If the last developer simply copied all layout files from the parent to the child, that is not ideal. You may want to run a diff to determine what’s changed
- If the theme doesn’t inherit, that means it uses the base/default theme as a fallback. Is the theme using local.xml and selective layout.xml overrides to make changes to the page? If the last developer simply copied all layout files from the base theme to the active theme, that is not ideal. You may want to run a diff to determine what’s changed.
Step 8: Audit Log Files / Log Tables
Look for errors / problems
If Magento has logging enabled (System -> Config -> Developer -> Enable Logging) chances are you will have a few files in var/log. I’m most interested in 2 files here:
- system.log – Used by Magento log all fatal errors / warnings / notices
- exception.log – Used by Magento when an exception is not caught
You will want to examine all entries in these files from the last 30 days. Once you go more than a month back, you may very well be viewing bugs that have already been fixed, etc. Copy a list of all critical bugs into a worddoc / excel spreadsheet. You want to have access to the bugs even if someone purges the logs.
Check log size
Check the size of the log files by running: du -h * in the var/log folder. If the sizes are over 100M for any files, you may want to consider setting up logrotate to clean up older entries in the files. Setting up logrotate is beyond the scope of this tutorial, but some useful information can be found here: https://www.digitalocean.com/community/tutorials/how-to-manage-log-files-with-logrotate-on-ubuntu-12-10
You will also want to run the below sql queries:
SELECT COUNT(*) FROM log_url; SELECT COUNT(*) FROM log_url_info; SELECT COUNT(*) FROM log_visitor; SELECT COUNT(*) FROM log_visitor_info;
There are some other log tables, but these are usually the big ones. If the size of these is massive, you’ll want to do one of two things:
#1) Truncate the tables
You can truncate them by running:
TRUNCATE log_url; TRUNCATE log_url_info; TRUNCATE log_visitor; TRUNCATE log_visitor_info;
Alternatively, a better (and more long-term) solution is to enable log cleaning. Do this in the admin panel via System -> Configuration -> System -> Enable Log Cleaning – the default of 180 days should be fine (you can also safely lower this number).
Step 9: Audit Performance
This one has 2 pieces. We’ll begin with using Google’s page speed insights to discover any issues. Go ahead and run your production site through the analyzer (make sure caching is enabled first). You should get a report like this:
This should give you a lot of handy information. Fixing each of the listed items is beyond the scope of the tutorial, but I will discuss a view here:
- Optimize Images – consider using a service like kraken.io or compressor.io
- Leverage Browser Caching – Use expires tags (tutorial here: https://gtmetrix.com/leverage-browser-caching.html)
- Reduce Server Response Time – More on this below
Most of these improvements are things you can do through apache/nginx or by updating the page content. However, Reduce Server Response Time may indicate more issues. This means your website is taking a long time produce the html that gets loaded by the browser. You have a few options:
- FPC (Full Page Cache) – if you’ve optimized everything in the code you can think of, try an FPC engine:
http://fishpig.co.uk/magento/extensions/full-page-cache/ – bolt fpc
https://github.com/GordonLesti/Lesti_Fpc – lesti fpc
- Audit the code – this one is trickier, but if you think a piece of custom functionality is slowing the site down, you will need to use the varien profiler and/or xdebug to trace through the code. I’ve actually provided a list of all the tools I sometimes use to fix this:
Tools for auditing php code:
- Varien Profiler
- NewRelic APM: http://newrelic.com/sp/application-monitoring
- Blackfire.io: https://blackfire.io/
I don’t have time to go into the how-to for each of these … they all have their pros/cons, however I recommend you start with the varien profiler and work your way down.
Step 10: Audit Extension / Custom Functionality
Returning to using n98, there is another handy command which will show us all extensions:
This will provide you with a full list of active extensions along with their version numbers and status (stable/beta/alpha), for example:
| lavi_news | 1.0.3 | stable |
| Custom_Login_Redirection | 0.1.0 | stable |
| Imedia_RewardPoints | 188.8.131.52 | stable |
| ImageRecycle_image_pdf_optimize | 1.0.6 | stable |
| Mzax_Emarketing | 0.4.10 | stable |
| Netgo_Nbanners | 1.0.4 | stable |
Is sample output from one of the sites I’m working on. All extensions will be located in app/code/community and app/code/local where you can investigate their source code. Now, not all extensions are created equal. Generally there are a few extensions (or custom modules) that are critical to a site’s business logic. You will want to pay special attention to these and spend time auditing each of them individually. I also recommend you ask the below list of questions:
- Are there any non-standard payment or shipping methods used on your site?
- Are there any warehouse / fulfillment integrations?
- Does your store have any type of subscription integration?
- Are you aware of any 3rd party API or FTP integrations?
The owner of the website may not know the specific module, but they should know whether or not the site is integrated with a third party.
Step 11: Audit Rewrite Conflicts
Alright! This is the last but not least section you’ll audit. Once again, n98 comes to the rescue with an easy command to figure this all out:
If there are any conflicts, you’ll see a shell printout similar to this:
You should go through each conflict and resolve it in one of several ways:
- Uncomment one modules rewrite in config.xml, and force the active rewrite to extend the disabled one.
- Create a new module that incorporates the modified code from both rewrites (and comment out the old rewrites in config.xml)
- For a full tutorial on this, see here: http://www.webshopapps.com/blog/2010/11/resolving-magento-extension-conflicts/
Ok! That was a lot of information. I hope this has given you a starting point in auditing a new site. Obviously you won’t have time to read every line of source code, so you’ll usually need to prioritize where the bulk of your time is spent during the audit, but make sure to spend at least 15 minutes on each section – you could be saving yourself 15 hours down the line!