AJAX slow for large sites (50k+ keywords)

[professionalseo]

Hello!

I'm currently running a Nginx setup with PHP-FPM. Very fast server, capable of sending 1000+ request per second. Also on a dedicated T1 line.

I am currently monitoring 50,000+ keywords. The results appear to be correct in the SERPs -- I modified the query to be precise with 10 results/page.

I had to create/reverse engineer the cron command, as the current cron.php file is broken (missing Javascript in HTTP mode, doesn't proc via lynx/wget via cron).

My issues are three fold at this point:

1) With such a large data set, the current tracking methodology does not return that the keyword was queried, regardless if it has a SERP ranking (to my knowledge, anyhow). I need to know that the keyword was checked, even if it doesn't rank. Is this possible with the current SP distribution?

2) With so many keywords, the keyword position summary takes 30+ seconds to generate. Is this possible to paginate?

3) AJAX appears to be "slow". Is this because of how many look ups it is doing to the mySQL?

[seopanel_forum]