Just wanted to share a conundrum we had over the last few days. It all started when a new back office area we are developing suddenly stopped displaying repeat regions correctly. Not all repeat regions were effected but quite a few ceased to display.

We run our own hosting for all of our developments. A little information on the environment. Running on Apache 2.4 with MYSQL 5.5 and PHP 5.5 and using WHM as a control panel on CentOS 6.7. ModSecurity (OWASP and Comodo rule-sets) is active alongside CSF. All sites are sat behind CloudFlare (we insist our clients subscribe to their services). So as you can see there are a few variables that could have played a part in this issue....

The first thing we checked were the database connections and they were fine. We inspected all the data sources and they were fine. We thought that a repeat rule violation in ModSecurity or CSF could have instigated a ban but no that area was also fine.. Maybe it was CloudFlare so we paused the service and checked again. No the problem persisted...

OK we thought lets dump the DB, all files, the whole shebang on to another server and create an exact copy without all the extras we have on our production environment... Still the same problem hmmm... The same issue was also apparent on our local test area. Now we are pulling our hair out as we want to enjoy Christmas and New Year like everyone else but we can't relax until we resolve the issues at hand. Next step create a page or two with new repeat regions. Throw them up and see if they work correctly.... YES! OK what the %&/&%$ is going on..?

At this point I said OK I am going to pull everything down and rebuild the servers. Drastic I know but sometimes you just have to do things like this to satisfy not only curiosity but also the concern that something was mis-configured or even worse had been compromised somewhere. While this process was going on I started going through log files from Apache, MYSQL, PHP, CSF, ModSec, and pretty much any log file I could gather. Half way through all the installs and updates I stumbled across something that made me cringe..... We had relocated a tiny part of our CDN to a new location and there was one JavaScript file that was related to the Back Office development (for language selection). It was moved as part of a tidying up process but we failed to check it had any connections to our development code base (silly).

We temporarily reinstated the file to its original location and after several hours re-deploying everything took a breath and loaded up the Back Office, signed in...... The console opened correctly and all the repeat regions were displayed! The relief was immediate as was my embarrassment.

What had happened was the last piece of JavaScript called before the DMX repeat regions had been added to to include the DMX JavaScript that controls the repeat regions! I had noticed this happens and should have checked but hindsight is a $%&/.

Below is the offending code:

<script type="text/javascript">
var lang = new Lang('en');
	lang.dynamic('pt', '../CDN_4_Extra_JS/js/langpack/Super_PT.json');   <----------- PROBLEM!
  /* dmxDataSet name "History_Active_Users" */
       jQuery.dmxDataSet(
         {"id": "History_Active_Users", "url": "../dmxDatabaseSources/History_Active_Users.php", "data": {"limit": "10"}, "dataSourceType": "database", "dataType": "jsonp", "update": "MM_changeProp('history_data_preloader','','display','none','DIV');"}
       );
  /* END dmxDataSet name "History_Active_Users" */

I just wanted to share this misadventure with everyone. Maybe it will save a lot of headaches for someone having the same problems... You certainly don't need to do what I had done.

Talk about overkill! I could have simply used the Developer Console inside the browser and would probably have spotted it....

Being Christmas I had one or two beverages.... I am blaming this whole event on that hahaha..... Moral of this story.......

NEVER DRINK AND CODE!

Happy 2016!