adding multithreading tests for batch processor

[aravinsiva]

Add's concurrency tests to ensure that force_flush() and shutdown() could be called concurrently without causing racing coniditions.

Addresses:
#392

[sjbrunst]

Not super familiar with this code, but what happens if spans are "created" by one thread while the other thread is shutting down? Is that something we should test as well?

[aravinsiva]

this is a great point. The only reason I didn't look into that in particular is because the two methods outlined in the issue were force_flush() and shutdown()

[aabmass]

What is the race condition we are testing here?

[lzchen]

@aravinsiva
Are you still working on this? Can you answer @aabmass 's question?

[aravinsiva]

@aravinsiva
Are you still working on this? Can you answer @aabmass 's question?

My mistake have been busy tackling another project. And it is currently testing that these can be called concurrently without causing a change in the end resulting spans. Feel free to let me know if this is not the correct approach to test what we want to be tested.

[aabmass]

@lzchen is this what you meant by:

I believe they are referring to the behaviour for shutdown and force_flush when they are called at the same time in different threads (e.g. multiple BatchSpanProcessors)

It looks like shutdown() would block until its done exporting

opentelemetry-python/opentelemetry-sdk/src/opentelemetry/sdk/trace/export/__init__.py

Lines 253 to 259 in b085f37

	def shutdown(self) -> None:
	# signal the worker thread to finish and then wait for it
	self.done = True
	with self.condition:
	self.condition.notify_all()
	self.worker_thread.join()
	self.span_exporter.shutdown()

So they wouldn't be called at the same time unless you actually did it in separate threads. Even if you did it in separate threads, the test might be flaky since you can't guarantee the order of execution.

[lzchen]

I think @Oberon00 's comment was referring to concurrency cases in general (calling force_flush manually while the span processor is exporting, calling shutdown while the processor is exporting, etc.).

[aravinsiva]

^ is this not tested in the way the test has been written currently @lzchen ? As I am calling force_flush and shutdown() manually. unless there is something I am not understanding.

[aabmass]

They are not called at the same time though, since your call to shutdown() doesn't return until it is done exporting

[codeboten]

Closing this PR, the work needed to ensure that the tracerprovider api is safe to use concurrently is still needed.