Latest update:
The site was restarted and met with initial slowness. Close monitoring
saw that traffic to djn was extremely high
and that caching population (the caching of project homepages) that
happens each post-restart began to effect
performance.
What occurs is that when the site is restarted the ALM cache is cleared,
to be then cached again with the first request
to each of those files. The result is that all these requests begin to
lock up the ALM tables which then in turn cause back ups
site wide.
So, the site was restarted after a scan was performed of the previous
day's data and the cache of all those projects was input manually.
The second restart leads us to where we are now, which is still a state
of an inordinate amount of traffic coming in to the
site. No conclusions can be yet drawn about the nature of the traffic,
but it is resulting in the continued slowness on site.
Needless to say, all hands are on deck investigating and working to
bring about standard performance levels.
I'll update here again when I have any new information.
Regards,
Eric