SI
SI
discoversearch

We've detected that you're using an ad content blocking browser plug-in or feature. Ads provide a critical source of revenue to the continued operation of Silicon Investor.  We ask that you disable ad blocking while on Silicon Investor in the best interests of our community.  If you are not using an ad blocker but are still receiving this message, make sure your browser's tracking protection is set to the 'standard' level.
SI - Site Forums : Silicon Investor - New feature discussion
 Public ReplyPrvt ReplyMark as Last ReadFilePrevious 10Next 10PreviousNext  
Recommended by:
J_W
Moonray
The Ox
To: Catfish who wrote (4413)9/11/2014 10:24:25 AM
From: SI Dmitry (code monkey)3 Recommendations  Read Replies (3) of 7352
 
Dear Catfish,

There is a query that runs hourly on the hour, which updates SI's Most Followed pages.

This is an expensive query with respect to the database performance - our tools that measure server use by CPU time consumed point towards this as being the cause of a brief period of slowness that occurs hourly for about one or two minutes.

After discovering and contemplating this, I began looking at my watch every time I sensed some lag in the server response times and found that like clockwork, we always appear to be at the very top of the hour when this occurs.

A solid longer term fix for this problem (and some others) is for us to fire up another VM to host a read-only, asynchronously updated mirror of our live database server and run such expensive queries against that read-only database. Moving the more intensive full-text searches over to this server would also relieve periodic stress the rest of the site sees.

We'll also need to look at how to optimize this query which has not been edited nor changed since I began overseeing the system administration here nearly two years ago. If this becomes too serious an issue before this longer-term fix, we might simply change the query to run daily (vs. hourly, at present) in the middle of the night, and update those pages to indicate the modified frequency.

Regards,

@Dima
Report TOU ViolationShare This Post
 Public ReplyPrvt ReplyMark as Last ReadFilePrevious 10Next 10PreviousNext  

HomeMailHotSubjectMarksPeopleMarksKeepersSettings
Terms Of UseContact UsCopyright/IP PolicyPrivacy PolicyAbout UsFAQAdvertise on SI
© 2025 Knight Sac Media.  Data provided by Twelve Data, Alpha Vantage, and CityFALCON News