[Python-Dev] PEP 567 v3

Yury Selivanov yselivanov.ml at gmail.com
Tue Jan 16 20:06:41 EST 2018


On Tue, Jan 16, 2018 at 7:45 PM, Guido van Rossum <guido at python.org> wrote:
> On Tue, Jan 16, 2018 at 4:37 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> On Tue, 16 Jan 2018 17:44:14 -0500
>> Yury Selivanov <yselivanov.ml at gmail.com> wrote:
>>
>> > Offloading execution to other threads
>> > -------------------------------------
>> >
>> > It is possible to run code in a separate OS thread using a copy
>> > of the current thread context::
>> >
>> >     executor = ThreadPoolExecutor()
>> >     current_context = contextvars.copy_context()
>> >
>> >     executor.submit(
>> >         lambda: current_context.run(some_function))
>>
>> Does it also support offloading to a separate process (using
>> ProcessPoolExecutor in the example above)?  This would require the
>> Context to support pickling.
>
>
> I don't think that's a requirement. The transparency between the two
> different types of executor is mostly misleading anyway -- it's like the old
> RPC transparency problem, which was never solved IIRC. There are just too
> many things you need to be aware of before you can successfully offload
> something to a different process.

I agree.

I think it would be a very fragile thing In practice: if you have even
one variable in the context that isn't pickleable, your code that uses
a ProcessPool would stop working.  I would defer Context pickleability
to 3.8+.

Yury


More information about the Python-Dev mailing list