Browse Source

提交

master
SisMaker 4 years ago
parent
commit
4e655ae2d7
3 changed files with 615 additions and 2 deletions
  1. +507
    -0
      arangodb/arangodb服务器选项.md
  2. +32
    -0
      arangodb/arangodb程序与工具.md
  3. +76
    -2
      arangodb/减少服务器的内存占用.md

+ 507
- 0
arangodb/arangodb服务器选项.md View File

@ -0,0 +1,507 @@
# 服务器用法
arangod [<option>]
可以将数据库目录指定为位置 第一个参数
arangod /path/to/datadir
或者显示作为命名参数
arangod --database.directory /path/to/datadir
所有其他参数都需要作为命名参数传递
两个连字符(--)后跟选项名称 然后 等号或者空格 最后是参数值 如果参数值包含了空格 还需要用引号引起来
= 两边 有空格
# 各种配置选项
## Global
Name Typ
check-configuration boolean
检查配置并退出 这是一个命令,不需要指定任何值。执行命令后,进程终止。
config string
The configuration file or ‘none’
Default: ""
configuration string
The configuration file or ‘none’
Default: ""
console boolean
Start a JavaScript emergency console
This option can be specified without value to enable it.
Default: false
daemon boolean
Background the server, running it as daemon
This option can be specified without value to enable it.
Default: false
default-language string
ISO-639 language code
Default: ""
define string…
Define key=value for a @key@ entry in config file
Default: []
dump-dependencies boolean
Dump dependency graph
This is a command, no value needs to be specified. The process terminates after executing the command.
dump-options boolean
Dump configuration options in JSON format
This is a command, no value needs to be specified. The process terminates after executing the command.
fortune boolean
Show fortune cookie on startup
This option can be specified without value to enable it.
Default: false
gid string
Switch to group-id after reading config files
Default: ""
hund boolean
Make ArangoDB bark on startup
This option can be specified without value to enable it.
Default: false
log string…
The global or topic-specific log level
Default: ["info","info"]
Deprecated in: v3.5.0
pid-file string
Pid-file in daemon mode
Default: ""
supervisor boolean
Background the server, starts a supervisor
This option can be specified without value to enable it.
Default: false
uid string
Switch to user-id after reading config files
Default: ""
version boolean
Reports the version and exits
This is a command, no value needs to be specified. The process terminates after executing the command.
working-directory string
Working directory in daemon mode
Default: "/var/tmp"
## Agency
Name Type Description
agency.activate boolean Activate agency
This option can be specified without value to enable it.
Default: false
Effective on Agents only.
agency.compaction-keep-size uint64 Keep as many indices before compaction point
Default: 50000
Effective on Agents only.
agency.compaction-step-size uint64 Step size between state machine compactions
Default: 1000
Effective on Agents only.
agency.disaster-recovery-id string Allows for specification of the id for this agent; dangerous option for disaster recover only!
Default: ""
Effective on Agents only.
agency.election-timeout-max double Maximum timeout before an agent calls for new election (in seconds)
Default: 5
Effective on Agents only.
agency.election-timeout-min double Minimum timeout before an agent calls for new election (in seconds)
Default: 1
Effective on Agents only.
agency.endpoint string… Agency endpoints
Default: []
Effective on Agents only.
agency.max-append-size uint64 Maximum size of appendEntries document (# log entries)
Default: 250
Effective on Agents only.
agency.my-address string Which address to advertise to the outside
Default: ""
Effective on Agents only.
agency.pool-size uint64 Number of agent pool
Default: 1
Effective on Agents only.
agency.size uint64 Number of agents
Default: 1
Effective on Agents only.
agency.supervision boolean Perform arangodb cluster supervision
This option can be specified without value to enable it.
Default: false
Effective on Agents only.
agency.supervision-frequency double Arangodb cluster supervision frequency (in seconds)
Default: 1
Effective on Agents only.
agency.supervision-grace-period double Supervision time, after which a server is considered to have failed (in seconds)
Default: 10
Effective on Agents only.
agency.supervision-ok-threshold double Supervision time, after which a server is considered to be bad [s]
Default: 5
Effective on Agents only.
agency.wait-for-sync boolean Wait for hard disk syncs on every persistence call (required in production)
This option can be specified without value to enable it.
Default: true
Effective on Agents only.
## ArangoSearch
Name Type Description
arangosearch.threads uint64 The exact number of threads to use for asynchronous tasks (0 == autodetect)
Default: 0
arangosearch.threads-limit uint64 Upper limit to the autodetected number of threads to use for asynchronous
## Audit
Name Type Description
audit.hostname string Enterprise Edition only
Hostname to use
Default: ""
audit.output string… Enterprise Edition only
Audit destinatio
## Backup
Name Type Description
backup.api-enabled string Whether the backup api is enabled (true) or not (false), or only enabled for superuser JWT (jwt)
Default: "true"
backup.local-path-prefix This option restricts any backup target to a given path prefix Default: "/"
## Cache
cache.rebalancing-interval uint64 Microseconds between rebalancing attempts
Default: 2000000
cache.size uint64 Size of cache in bytes
Default: dynamic (e.g. 33252951040)
## Cluster
Name Type Description
cluster.agency-endpoint string… Agency endpoint to connect to
Default: []
Effective on Coordinators and DB-Servers only.
cluster.create-waits-for-sync-replication boolean Active coordinator will wait for all replicas to create collection
This option can be specified without value to enable it.
Default: true
Effective on Coordinators and DB-Servers only.
cluster.default-replication-factor uint32 Default replication factor for non-system collections
Default: 0
Effective on Coordinators only.
Introduced in: v3.6.0
cluster.force-one-shard boolean Force one-shard mode for all new collections
This option can be specified without value to enable it.
Default: false
Effective on Coordinators only.
Introduced in: v3.6.0
cluster.index-create-timeout double Amount of time (in seconds) the coordinator will wait for an index to be created before giving up
Default: 3600
Effective on Coordinators only.
cluster.max-number-of-shards uint32 Maximum number of shards when creating new collections (0 = unrestricted)
Default: 1000
Effective on Coordinators only.
Introduced in: v3.5.1
cluster.max-replication-factor uint32 Maximum replication factor for new collections (0 = unrestricted)
Default: 10
Effective on Coordinators only.
Introduced in: v3.6.0
cluster.min-replication-factor uint32 Minimum replication factor for new collections
Default: 1
Effective on Coordinators only.
Introduced in: v3.6.0
cluster.my-address string This server’s endpoint (cluster internal)
Default: ""
Effective on Coordinators and DB-Servers only.
cluster.my-advertised-endpoint string This server’s advertised endpoint (e.g. external IP address or load balancer, optional)
Default: ""
Effective on Coordinators and DB-Servers only.
cluster.my-role string This server’s role
Default: ""
cluster.require-persisted-id boolean If set to true, then the instance will only start if a UUID file is found in the database on startup. Setting this option will make sure the instance is started using an already existing database directory and not a new one. For the first start, the UUID file must either be created manually or the option must be set to false for the initial startup
This option can be specified without value to enable it.
Default: false
cluster.resign-leadership-on-shutdown boolean Create resign leader ship job for this dbsever on shutdown
This option can be specified without value to enable it.
Default: false
Effective on DB-Servers only.
cluster.synchronous-replication-timeout-factor double All synchronous replication timeouts are multiplied by this factor
Default: 1
cluster.synchronous-replication-timeout-minimum double All synchronous replication timeouts will be at least this value (in seconds)
Default: 30
cluster.synchronous-replication-timeout-per-4k double All synchronous replication timeouts are increased by this amount per 4096 bytes (in seconds)
Default: 0.1
cluster.system-replication-factor uint32 Default replication factor for system collections
Default: 2
Effective on Coordinators only.
cluster.upgrade string Perform a cluster upgrade if necessary (auto = perform upgrade and shut down only if --database.auto-upgrade true is set, disable = never perform upgrade, force = always perform an upgrade and shut down, online = always perform an upgrade but don’t shut down)
Default: "auto"
Possible values: “auto”, “disable”, “force”, “online”
cluster.write-concern uint32 Write concern used for writes to new collections
Default: 1
Effective on Coordinators only.
Introduced in: v3.6.0
## Database
Name Type Description
database.auto-upgrade boolean Perform a database upgrade if necessary
This option can be specified without value to enable it.
Default: false
database.check-version boolean Checks the versions of the database and exit
This is a command, no value needs to be specified. The process terminates after executing the command.
database.directory string Path to the database directory
Default: ""
database.force-sync-properties boolean Force syncing of collection properties to disk, will use waitForSync value of collection when turned off
This option can be specified without value to enable it.
Default: true
database.ignore-datafile-errors boolean Load collections even if datafiles may contain errors
This option can be specified without value to enable it.
Default: false
database.init-database boolean Initializes an empty database
This is a command, no value needs to be specified. The process terminates after executing the command.
database.password string Initial password of root user
Default: ""
database.required-directory-state string Required state of database directory at startup (non-existing: database directory must not exist, existing: database directory must exist, empty: database directory must exist but be empty, populated: database directory must exist and contain specific files already, any: any state allowed)
Default: "any"
Possible values: “any”, “empty”, “existing”, “non-existing”, “populated”
database.restore-admin boolean Resets the admin users and sets a new password
This is a command, no value needs to be specified. The process terminates after executing the command.
database.throw-collection-not-loaded-error boolean Throw an error when accessing a collection that is still loading
This option can be specified without value to enable it.
Default: false
Deprecated in: v3.7.0
database.upgrade-check boolean Skip a database upgrade
This option can be specified without value to enable it.
Default: true
database.wait-for-sync boolean Default wait-for-sync behavior, can be overwritten when creating a collection
This option can be
## Foxx
Name Type Description
foxx.api boolean Enables Foxx management REST APIs
This option can be specified without value to enable it.
Default: true
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
foxx.queues boolean Enable Foxx queues
This option can be specified without value to enable it.
Default: true
Effective on Coordinators and Single Servers only.
foxx.queues-poll-interval double Poll interval (in seconds) for Foxx queue manager
Default: 1
Effective on Coordinators and Single Servers only.
foxx.store boolean Enables Foxx store in web interface
This option can be specified without value to enable it.
Default: true
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
# Frontend
Name Type Description
frontend.proxy-request-check boolean Enable proxy request checking
This option can be specified without value to enable it.
Default: true
Effective on Coordinators and Single Servers only.
frontend.trusted-proxy string… List of proxies to trust (may be IP or network). Make sure --frontend.proxy-request-check is enabled
Default: []
Effective on Coordinators and Single Servers only.
frontend.version-check boolean Alert the user if new versions are available
This option can be specified without value to enable it.
Default: true
Effective on Coordinators and Single Servers only.
## HTTP
Name Type Description
http.allow-method-override boolean Allow HTTP method override using special headers
This option can be specified without value to enable it.
Default: false
http.hide-product-header boolean Do not expose “Server: ArangoDB” header in HTTP responses
This option can be specified without value to enable it.
Default: false
http.keep-alive-timeout double Keep-alive timeout in seconds
Default: 300
http.trusted-origin string… Trusted origin URLs for CORS requests with credentials
Default: []
# JavaScript
Name Type Description
javascript.allow-admin-execute boolean For testing purposes allow ‘_admin/execute’, NEVER enable on production
This option can be specified without value to enable it.
Default: false
Effective on Coordinators and Single Servers only.
javascript.allow-external-process-control boolean Allow execution and control of external processes from within JavaScript actions
This option can be specified without value to enable it.
Default: false
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.allow-port-testing boolean Allow testing of ports from within JavaScript actions
This option can be specified without value to enable it.
Default: false
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.app-path string Directory for Foxx applications
Default: "./js/apps"
Effective on Coordinators and Single Servers only.
javascript.copy-installation boolean Copy contents of ‘javascript.startup-directory’ on first start
This option can be specified without value to enable it.
Default: false
Effective on Coordinators and Single Servers only.
javascript.enabled boolean Enable the V8 JavaScript engine
This option can be specified without value to enable it.
Default: true
Effective on Coordinators and Single Servers only.
javascript.endpoints-blacklist string… Endpoints that cannot be connected to via @arangodb/request module in JavaScript actions if not whitelisted
Default: []
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.endpoints-whitelist string… Endpoints that can be connected to via @arangodb/request module in JavaScript actions
Default: []
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.environment-variables-blacklist string… Environment variables that will be inaccessible in JavaScript if not whitelisted
Default: []
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.environment-variables-whitelist string… Environment variables that will be accessible in JavaScript
Default: []
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.files-whitelist string… Filesystem paths that will be accessible from within JavaScript actions
Default: []
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.gc-frequency double JavaScript time-based garbage collection frequency (each x seconds)
Default: 60
Effective on Coordinators and Single Servers only.
javascript.gc-interval uint64 JavaScript request-based garbage collection interval (each x requests)
Default: 2000
Effective on Coordinators and Single Servers only.
javascript.harden boolean Disables access to JavaScript functions in the internal module: getPid() and logLevel()
This option can be specified without value to enable it.
Default: false
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.module-directory string… Additional paths containing JavaScript modules
Default: ["./enterprise/js"]
Effective on Coordinators and Single Servers only.
javascript.script string… Run scripts and exit
Default: []
javascript.script-parameter string… Script parameter
Default: []
javascript.startup-directory string Path to the directory containing JavaScript startup scripts
Default: "./js"
Effective on Coordinators and Single Servers only.
javascript.startup-options-blacklist string… Startup options whose names match this regular expression will not be exposed (if not whitelisted) to JavaScript actions
Default: []
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.startup-options-whitelist string… Startup options whose names match this regular expression will be whitelisted and exposed to JavaScript
Default: []
Effective on Coordinators and Single Servers only.
Introduced in: v3.5.0
javascript.v8-contexts uint64 Maximum number of V8 contexts that are created for executing JavaScript actions
Default: 0
Effective on Coordinators and Single Servers only.
javascript.v8-contexts-max-age double Maximum age for each V8 context (in seconds) before it is disposed
Default: 60
Effective on Coordinators and Single Servers only.
javascript.v8-contexts-max-invocations uint64 Maximum number of invocations for each V8 context before it is disposed
Default: 0
Effective on Coordinators and Single Servers only.
javascript.v8-contexts-minimum uint64 Minimum number of V8 contexts that keep available for executing JavaScript actions
Default: 0
Effective on Coordinators and Single Servers only.
javascript.v8-max-heap uint64 Maximal heap size (in MB)
Default: 3072
javascript.v8-options string… Options to pass to v8
Default: []
## LDAP
Name Type Description
ldap.allow-offline boolean Enterprise Edition only
If a refresh attempt fails to connect to the LDAP server, continue with the cached authentication data
This option can be specified without value to enable it.
Default: false
ldap.async-connect boolean Enterprise Edition only
Whether or not the connection to the LDAP library will be done asynchronously
This option can be specified without value to enable it.
Default: false
ldap.basedn string Enterprise Edition only
Ldap basedn, eg. dc=example,dc=com
Default: ""
ldap.binddn string Enterprise Edition only
Ldap binddn, eg. cn=admin,dc=example,dc=com
Default: ""
ldap.bindpasswd string Enterprise Edition only
Ldap bindpassword, eg. admin
Default: ""
ldap.debug boolean Enterprise Edition only
Turn on internal OpenLDAP library output (warning: will print to stdout)
This option can be specified without value to enable it.
Default: false
ldap.enabled boolean Enterprise Edition only
Enable LDAP
This option can be specified without value to enable it.
Default: false
ldap.network-timeout double Enterprise Edition only
Timeout value (in seconds) after which network operations following the initial connection return in case of no activity (a value of 0 means default timeout)
Default: 0
ldap.port uint16 Enterprise Edition only
Port to use
Default: 389
ldap.prefix string Enterprise Edition only
Ldap prefix, eg. uid= xor dn= xor cn=
Default: ""
ldap.referrals boolean Enterprise Edition only
Whether or not the LDAP library should implicitly chase referrals
This option can be specified without value to enable it.
Default: false
ldap.refresh-rate double Enterprise Edition only
Refresh user settings after this time (in seconds)
Default: 300
ldap.restart boolean Enterprise Edition only
Whether or not the LDAP library should implicitly restart connections
This option can be specified without value to enable it.
Default: false
ldap.retries uint32 Enterprise Edition only
Number of tries to attempt connecting to the LDAP server. Setting it to values greater than one will retry connecting in case the LDAP server is unavailable or denies the connection
Default: 1
ldap.roles-attribute-name string Enterprise Edition only
Ldap attributename where the role are located.
Default: ""
ldap.roles-exclude string Enterprise Edition only
Regexp to exclude groups. Leave empty to exclude none.
Default: ""
ldap.roles-include string Enterprise Edition only
Regexp to include groups. Leave empty to include all.
Default: ""
ldap.roles-search string Enterprise Edition only
Ldap search for roles; ‘{USER}’ is replaced by the ‘dn’ of the user.
Default: ""
ldap.roles-transformation string… Enterprise Edition only
Regexp to normalizer role name, e.g. ‘/^ (.[^ ]])*/$2/’
Default: []
ldap.search-attribute string Enterprise Edition only
Ldap search attribute, eg. uid
Default: "uid"
ldap.search-filter string Enterprise Edition only
Ldap search filter, eg. (objectClass=simpleSecurityObject)
Default: "objectClass=*"
ldap.search-scope string Enterprise Edition only
Ldap search scope, one of base, one, sub
Default: "sub"
ldap.serialize-timeout double Enterprise Edition only
Maximum amount of time (in seconds) that will be waited for the serialization mutex
Default: 5
ldap.serialized boolean Enterprise Edition only
Whether or not calls into the LDAP library should be serialized. This option can be used to work around thread-unsafe LDAP library functionality
This option can be specified without value to enable it.
Default: false
ldap.server string Enterprise Edition only
Server to use
Default: ""
ldap.suffix string Enterprise Edition only
Ldap suffix, eg. ,dc=example,dc=com
Default: ""
ldap.superuser-role string Enterprise Edition only
Role mapping to the super-users
Default: ""
ldap.timeout double Enterprise Edition only
Timeout value (in seconds) for synchronous LDAP API calls (a value of 0 means default timeout)
Default: 0
ldap.tls boolean Enterprise Edition only
Enable TLS
This option can be specified without value to enable it.
Default: false
ldap.tls-cacert-dir string Enterprise Edition only
Ldap tls cacert dir
Default: ""
ldap.tls-cacert-file string Enterprise Edition only
Ldap tls cacert file
Default: ""
ldap.tls-cert-check-strategy string Enterprise Edition only
Ldap tls cert check strategy, one of never, hard, demand, allow, try
Default: "hard"
ldap.tls-version string Enterprise Edition only
Ldap tls version, one of 1.0, 1.1, 1.2
Default: "1.2"
ldap.url string Enterprise Edition only
Ldap url, eg. ldap://example.com:389/dc=example,dc=com?uid?sub
Default: ""

+ 32
- 0
arangodb/arangodb程序与工具.md View File

@ -0,0 +1,32 @@
# 完成的arangodb软件包工具
二进制名称 简要描述;简介
arangod ArangoDB服务器。该服务器程序旨在作为守护进程/服务运行,以通过TCP / HTTP与服务器建立各种客户端连接。它还提供了一个Web界面。
arangosh ArangoDB shell。客户端,该客户端实现读评估打印循环(REPL)并提供访问和管理ArangoDB服务器的功能。
arangodb ArangoDB Starter,可轻松部署ArangoDB实例。
arangodump 用于创建 ArangoDB数据库备份的工具。
arangorestore 将备份加载回ArangoDB数据库的工具。
arangobackup 工具来进行热备份操作上ArangoDB安装。
arangoimport ArangoDB服务器的批量导入器。它支持JSON和CSV。
arangoexport ArangoDB服务器的批量导出器。它支持JSON,CSV和XML。
arangobench 基准和测试工具。它可以用于性能和服务器功能测试。
arangoinspect 收集服务器设置信息的检查工具。
arangovpack 将VelocyPack数据转换为JSON的实用程序。 \
客户端软件包附带了一部分程序和工具:
arangosh
arangoimport
arangoexport
arangodump
arangorestore
arangobackup
arangobench
arangoinspect
arangovpack
# 单独提供其他工具
单独提供的其他工具:
名称 简要描述;简介
Foxx CLI 用于管理和开发Foxx服务的命令行工具
kube-arangodb 运营商来管理Kubernetes部署
Oasisctl 用于管理ArangoDB Oasis的命令行工具

+ 76
- 2
arangodb/减少服务器的内存占用.md View File

@ -80,6 +80,80 @@
--javascript.v8-max-heap 512
这会将V8隔离株的数量限制为两个。所有与JavaScript相关的请求都将排队,直到其中一个隔离可用于新任务为止。它还将堆大小限制为512 MB,因此在最坏的情况下,两个V8上下文组合使用的内存不能超过1 GB。
# V8 for the Desperate
# V8 for the Desperate
除非有充分的理由,否则不要使用以下设置,例如性能不重要的本地开发系统或硬件资源非常有限的嵌入式系统!
--javascript.v8-contexts 1
--javascript.v8-max-heap 256
您可以将V8的内存使用量减少到256 MB,而只有一个线程。例如,某些操作可能会因为某些操作耗尽内存而中止,例如在Web界面中。另外,JavaScript请求将被一一执行。
如果您的内存非常紧张,并且确定不需要V8,则可以将其完全禁用:
--javascript.enabled false
--foxx.queues false
因此,以下功能将不可用:
Web界面的后端部分
Foxx应用
Foxx队列
GraphQL
基于JavaScript的交易
用户定义的AQL功能
请注意,可以为群集中的DB-Server和Agency节点禁用JavaScript / V8,而没有这些限制。它们适用于单个服务器实例。它们也适用于Coordinator节点,但是您不应在Coordinators上禁用V8,因为某些集群操作依赖于此。
# CPU使用率固定链接
我们不能真正减少CPU使用率,但是可以并行运行的线程数。同样,除非有很好的理由(例如嵌入式系统),否则您不应这样做。请注意,这将限制并发请求的性能,这对于只有您作为用户的本地开发系统来说可能是可以的。
可以通过以下方式限制后台线程的数量:
--arangosearch.threads-limit 1
--rocksdb.max-background-jobs 4
--server.maintenance-threads 2
--server.maximal-threads 4
--server.minimal-threads 1
通常,选择线程数以适合机器。但是,每个线程至少需要8 MB的堆栈内存。通过牺牲一些用于并行执行的性能,可以减少此性能。
此选项将使日志记录同步:
--log.force-direct true
除非您仅记录错误和警告,否则不建议这样做。
# 缓存分配说明
例子固定链接
通常,您应该在标准服务器上调整读取缓冲区和边缘缓存。如果您有图用例,则应该使用更大的边缘缓存。例如,在读取缓冲区和边缘缓存之间以50:50的比例分配内存。如果没有边缘,则使用最小边缘缓存,并将大部分内存用于读取缓冲区。
例如,如果您有一台具有40 GB内存的计算机,并且想要将ArangoDB的内存限制为20 GB,则如果使用图形功能,则将10 GB用于边缘缓存,将10 GB用于读取缓冲区。
请记住,在查询执行期间,额外的内存将临时用于查询结果。如果您的内存不足,则可能需要每个7 GB。
如果您具有嵌入式系统或开发笔记本电脑,则可以使用上述所有设置来进一步减少内存占用。对于正常操作,特别是生产,不建议使用这些设置。
# Linux系统配置
ArangoDB的主要部署目标是Linux。正如您所了解的,ArangoDB及其内部结构在内存上的工作量很大。因此,至关重要的是要知道ArangoDB和Linux内核如何就此进行交互。linux内核提供了几种如何管理内存的模式。您可以通过proc文件系统,该文件/etc/sysctl.conf或/etc/sysctl.conf.d/系统在引导时将其应用于内核设置的文件来影响此设置。如下所述的设置适用于sysctl基础结构,这意味着它们以映射到proc文件系统/proc/sys/vm/overcommit_memory。
甲vm.overcommit_memory的设定2可以引起在一些环境问题结合捆绑内存分配器ArangoDB附带(jemalloc)。
分配器要求内核提供连续的内存块,这些内存块也映射到磁盘上的块。这是代表服务器进程(arangod)完成的。该过程可能会长时间使用一个块中的某些块,但仅在短时间内使用其他块,因此会释放内存。然后由分配器将释放的部分返回给内核。因为它只能返回连续的内存块,所以它必须将大块拆分为多个小块,然后可以返回未使用的块。
随着vm.overcommit_memory内核设置值2,分配器可能与拆分现有的内存映射,这使得故障数量 的arangod服务器进程随时间增长的内存映射。这可能导致内核拒绝将更多内存分配给Arangod进程,即使有更多物理内存可用。内核将最多只授予vm.max_map_count 每个进程的内存映射,在许多Linux环境中默认为65530。
运行vm.overcommit_memory设置为2的 jemalloc时的另一个问题是,对于某些工作负载,Linux内核随着已提交的内存跟踪的 内存量也会随着时间的推移而增长,而从未减少。最终, arangod可能仅由于达到配置的过量使用限制(物理RAM * overcommit_ratio+交换空间)而无法获得更多的内存。
解决的办法是 修改的值vm.overcommit_memory 从2至任一0或1。这将解决这两个问题。无论使用过量提交设置如何,使用jemalloc时,我们仍然观察到虚拟内存消耗不断增加,但是实际上这不会引起任何问题。0是Linux内核的默认值,也是我们建议的设置。
为了完整起见,让我们还提到另一种解决问题的方法:使用其他内存分配器。这要求从源代码编译ArangoDB而不使用jemalloc(-DUSE_JEMALLOC=Off在对cmake的调用中)。使用系统的libc分配器,您应该会看到相当稳定的内存使用情况。我们还尝试了另一个分配器,恰好是来自的分配器,libmusl随着时间的推移,这也显示出相当稳定的内存使用情况。阻碍我们更改捆绑分配器的原因在于,这是一次微不足道的更改,因为jemalloc对于大型多线程进程具有非常好的性能特征。
测试减少的I / O缓冲区的效果
效果图
15:50 –开始更大的进口
16:00 –一次开始写入〜60 KB大小的文档
16:45 –添加类似的第二作家
16:55 –使用上面建议的RocksDB写缓冲区配置重新启动ArangoDB
17:20 –缓冲区已满,写入性能下降
17:38 – WAL轮播
您在上面的性能图中看到的是限制写缓冲区的后果。在达到90%的写缓冲区填充率之前,服务器几乎可以遵循加载模式一段时间,但代价是不断增加缓冲区。一旦RocksDB达到90%的缓冲区填充率,它将显着将负载限制在〜50%。根据上游文档,这是预期的 :
如果可变存储器的总大小超过限制的90%,则会触发刷新。如果实际内存超出限制,则即使可变存储器的总大小小于90%,也可能触发更积极的刷新。
由于我们仅测量磁盘I / O字节,因此看不到文档保存操作的请求时间也增加了一倍。

Loading…
Cancel
Save