sphinx占用大量內存,甚至無法啟動怎麽辦?

王朝幹貨·作者佚名  2023-07-10

在sphinx.conf裏的source配置項裏,使用sql_attr_string時,會使索引文件增大許多,而默認配置會把這些都加載到內存裏,導致內存占用過大。(使用sql_attr_string的好處是不用再去查詢數據庫了。)

解決辦法是在index配置項裏,加上ondisk_attrs = pool。

在老版本sphinx裏是使用docinfo = extern,但在2.2.1-beta版本後docinfo作廢了,使用ondisk_attrs來配置存儲模式。

ondisk_attrs有3個選項,0、1、pool。

0是默認值,所有內容都加載進內存;1,所有內容都放在硬盤上;pool,string, MVA, and JSON attributes (sps, spm files)等放在硬盤上,數字等加載進內存。

0消耗內存最大(幾個G),速度最快(毫秒級);1消耗內存最小(幾百M),速度最慢(秒級);pool選項親測接近0的速度和1的內存使用量,所以沒有懸念的選擇pool。

下面是官方文檔說明:

ondisk_attrs

Allows for fine-grain control over how attributes are loaded into memory when using indexes with external storage. It is now possible (since version 2.2.1-beta) to keep attributes on disk. Although, the daemon does map them to memory and the OS loads small chunks of data on demand. This allows use of docinfo = extern instead of docinfo = inline, but still leaves plenty of free memory for cases when you have large collections of pooled attributes (string/JSON/MVA) or when you're using many indexes per daemon that don't consume memory. It is not possible to update attributes left on disk when this option is enabled and the constraint of 4Gb of entries per pool is still in effect.

Note that this option also affects RT indexes. When it is enabled, all atribute updates will be disabled, and also all disk chunks of RT indexes will behave described above. However inserting and deleting of docs from RT indexes is still possible with enabled ondisk_attrs.

Possible values:

0 - disabled and default value, all attributes are loaded in memory (the normal behaviour of docinfo = extern)

1 - all attributes stay on disk. Daemon loads no files (spa, spm, sps). This is the most memory conserving mode, however it is also the slowest as the whole doc-id-list and block index doesn't load.

pool - only pooled attributes stay on disk. Pooled attributes are string, MVA, and JSON attributes (sps, spm files). Scalar attributes stored in docinfo (spa file) load as usual.

This option does not affect indexing in any way, it only requires daemon restart.

Example:

ondisk_attrs = pool #keep pooled attributes on disk

 
 
免責聲明:本文為網絡用戶發布,其觀點僅代表作者個人觀點,與本站無關,本站僅提供信息存儲服務。文中陳述內容未經本站證實,其真實性、完整性、及時性本站不作任何保證或承諾,請讀者僅作參考,並請自行核實相關內容。
 
© 2005- 王朝網路 版權所有 導航