Replication hangs due to invalidating query cache entries (table) (no replies)

Dear MySQL Gurus,

These days I am facing strange issues with the replication. Here is the latest issue that I am facing. I am using MySQL 5.1.47 (64bit) on CentOs 64 bit. I have one master server and 4 slave servers. One of the slave has issues with Replication and it get behind sometimes 4000 seconds behind master. When I check show full processlist, I noticed the invalidating query cache entries (table) is taking almost same time which is reported by replication Seconds_Behind_Master. I even turned off Query_cache_type but still facing this issue. Following are server parameters

+------------------------------+----------+
| Variable_name | Value |
+------------------------------+----------+
| have_query_cache | YES |
| query_cache_limit | 2097152 |
| query_cache_min_res_unit | 4096 |
| query_cache_size | 67108864 |
| query_cache_type | OFF |
| query_cache_wlock_invalidate | OFF |
+------------------------------+----------+

I have some tables which don't have any index on master but they do have indexes on slave sides. Can anybody please guide me how to fix this issue?

Thanking you in advance!