首页 > 其他 > 详细

:setting:`task_soft_time_limit` celery 异步任务 执行时间限制 内存限制

时间:2020-07-02 09:19:54      阅读:157      评论:0      收藏:0      [点我收藏+]

 

 

https://docs.celeryproject.org/en/stable/userguide/configuration.html?highlight=control_exchange#new-lowercase-settings

 

New lowercase settings

Version 4.0 introduced new lower case settings and setting organization.

The major difference between previous versions, apart from the lower case names, are the renaming of some prefixes, like celery_beat_ to beat_celeryd_ to worker_, and most of the top level celery_ settings have been moved into a new task_ prefix.

Note

Celery will still be able to read old configuration files, so there’s no rush in moving to the new settings format. Furthermore, we provide the celery upgrade command that should handle plenty of cases (including Django).

 

 

https://github.com/celery/celery/blob/master/docs/userguide/workers.rst#id198

Time Limits

.. versionadded:: 2.0

pool support: prefork/gevent

Soft, or hard?

The time limit is set in two values, soft and hard. The soft time limit allows the task to catch an exception to clean up before it is killed: the hard timeout isn‘t catch-able and force terminates the task.

A single task can potentially run forever, if you have lots of tasks waiting for some event that‘ll never happen you‘ll block the worker from processing new tasks indefinitely. The best way to defend against this scenario happening is enabling time limits.

The time limit (--time-limit) is the maximum number of seconds a task may run before the process executing it is terminated and replaced by a new process. You can also enable a soft time limit (--soft-time-limit), this raises an exception the task can catch to clean up before the hard time limit kills it:

from myapp import app
from celery.exceptions import SoftTimeLimitExceeded

@app.task
def mytask():
    try:
        do_work()
    except SoftTimeLimitExceeded:
        clean_up_in_a_hurry()

Time limits can also be set using the :setting:`task_time_limit` / :setting:`task_soft_time_limit` settings.

 

 

Changing time limits at run-time

.. versionadded:: 2.3

broker support: amqp, redis

There‘s a remote control command that enables you to change both soft and hard time limits for a task — named time_limit.

Example changing the time limit for the tasks.crawl_the_web task to have a soft time limit of one minute, and a hard time limit of two minutes:

>>> app.control.time_limit(‘tasks.crawl_the_web‘,
                           soft=60, hard=120, reply=True)
[{‘worker1.example.com‘: {‘ok‘: ‘time limits set successfully‘}}]

Only tasks that starts executing after the time limit change will be affected.

 

Time Limits

.. versionadded:: 2.0

pool support: prefork/gevent

Soft, or hard?

The time limit is set in two values, soft and hard. The soft time limit allows the task to catch an exception to clean up before it is killed: the hard timeout isn‘t catch-able and force terminates the task.

A single task can potentially run forever, if you have lots of tasks waiting for some event that‘ll never happen you‘ll block the worker from processing new tasks indefinitely. The best way to defend against this scenario happening is enabling time limits.

The time limit (--time-limit) is the maximum number of seconds a task may run before the process executing it is terminated and replaced by a new process. You can also enable a soft time limit (--soft-time-limit), this raises an exception the task can catch to clean up before the hard time limit kills it:

from myapp import app
from celery.exceptions import SoftTimeLimitExceeded

@app.task
def mytask():
    try:
        do_work()
    except SoftTimeLimitExceeded:
        clean_up_in_a_hurry()

Time limits can also be set using the :setting:`task_time_limit` / :setting:`task_soft_time_limit` settings.

Note

Time limits don‘t currently work on platforms that don‘t support the :sig:`SIGUSR1` signal.

 

 

 

 

 

 

 

 

 

:setting:`task_soft_time_limit` celery 异步任务 执行时间限制 内存限制

原文:https://www.cnblogs.com/rsapaper/p/13221419.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!