[PATCH 3 of 4 celerytask] Add a new celerytask-log-dir option

Denis Laxalde denis.laxalde at logilab.fr
Mon Jun 25 09:33:26 CEST 2018


Philippe Pepiot a écrit :
> # HG changeset patch
> # User Philippe Pepiot <philippe.pepiot at logilab.fr>
> # Date 1529510234 -7200
> #      Wed Jun 20 17:57:14 2018 +0200
> # Node ID 82130ff7c6d5d8e733c34881bc08a6b066c6a962
> # Parent  8079245626055b42cee6be8de1fb41fbc15a4fb6
> # Available At https://hg.logilab.org/review/cubes/celerytask
> #              hg pull https://hg.logilab.org/review/cubes/celerytask -r 82130ff7c6d5
> Add a new celerytask-log-dir option
> 
> We're going to introduce a new file based logger for celery task logs.
> Add a new 'celerytask-log-dir' option that is used to store those logs files.
> 
> The CUBICWEB_CELERYTASK_LOGDIR option has been introduced in
> cw-celerytask-helpers for non-cubicweb workers (i.e. celery workers that
> doesn't require a cubicweb connexion).
> 
> We must ensure that both options are consistent and set the the appropriate
> value, so a startup hook ensure the configuration is correct.
> 
> diff --git a/hooks.py b/hooks.py
> --- a/hooks.py
> +++ b/hooks.py
> @@ -18,6 +18,9 @@
>  """cubicweb-celerytask specific hooks and operations"""
>  from __future__ import print_function
>  
> +import errno
> +import os.path
> +
>  from celery import current_app
>  import celery.task.control
>  
> @@ -71,3 +74,36 @@ class CeleryTaskDeletedHook(Hook):
>          op = DeleteCeleryTaskOp.get_instance(self._cw)
>          for task in self.entity.child_tasks():
>              op.add_data(task.task_id)
> +
> +
> +class CeleryTaskStartupHook(Hook):
> +    """Initialize CUBICWEB_CELERYTASK_LOGDIR"""
> +    __regid__ = 'celerytask.server-startup-hook'
> +    events = ('server_startup', 'server_maintenance')
> +
> +    def __call__(self):
> +        self.setup_celerytask_logdir(self.repo.vreg.config)
> +
> +    @staticmethod
> +    def setup_celerytask_logdir(config):
> +        celery_logdir = current_app.conf.get('CUBICWEB_CELERYTASK_LOGDIR')
> +        logdir = config['celerytask-log-dir']
> +        if celery_logdir and logdir and celery_logdir != logdir:
> +            raise RuntimeError((

There's a ConfigurationError exception type in cubicweb, perhaps it'd be
appropriate here?

Otherwise, the patch LGTM.

> +                "You misconfigured your application by setting two different "
> +                "celerytask log directories: "
> +                "{} in celeryconfig and {} in cubicweb all-in-one.conf"
> +            ).format(celery_logdir, logdir))
> +        elif celery_logdir and not logdir:
> +            logdir = config['celerytask-log-dir'] = celery_logdir
> +        elif not celery_logdir:
> +            if not logdir:
> +                logdir = os.path.join(config.appdatahome, 'logs')
> +            current_app.conf['CUBICWEB_CELERYTASK_LOGDIR'] = logdir
> +            config['celerytask-log-dir'] = logdir
> +        # ensure directory exist
> +        try:
> +            os.makedirs(logdir)
> +        except OSError as exc:
> +            if exc.errno != errno.EEXIST:
> +                raise


More information about the cubicweb-devel mailing list