Tracing database dependancies in django

[arkataev]

How can one trace a django database dependancy to Azure Insights?
For instance, if I have a basic call to Model.objects.all() how this might be traced? And what data can be traced?

Does it make sense to use a opencensus-ext-mysql in order to send traces to Azure Insights?
Are there any examples for this?

p.s unfortunately, there are no any READMEs for those sql-related plugins.

[reyang]

I believe #775 is trying to address this.

[arkataev]

This one looks good, but it utilises Django2 features. Is it possible to get similar results using some of the existing extensions applied to django 1.11 ?

[tpyo]

Yes, you can use the Django integration with the MySQL integration to get query tracing in <= Django 1.11 (provided you're using mysql-connector).

You would only need to load the MySQL integration with config_integration.trace_integrations(['mysql']), as long as the tracer is initialised by the Django middleware it should work as expected.

[arkataev]

Big Thanks for the tip, @tpyo! I'll check this out.

[arkataev]

I've tried config_integration.trace_integrations(['pymysql']) and seems its OK. But, I've noticed that there's only two main span kinds for now: client and server that resolves to HTTP and INPROC dependancy types accordingly in Azure Insights. But it looks like there's also SQL-dependancy type supported by Azure Insights API. Do you plan to implement this type in further library releases?

[victoraugustolls]

This is related to #737, maybe if we internally changed the trace parameters like I did in #738, we could check for SQL as a dependency type? @reyang

[victoraugustolls]

@reyang what do you think? AzureExporter can check for those tags (even though they are OpenTelemetry specs)

[reyang]

@victoraugustolls I agree with you.

[victoraugustolls]

@reyang I have a local version that I use with this behavior. Will adapt it and add tests so I can open a PR.

[jordanorc]

I'm facing problems to tracing database queries. In the middleware.py file, execute_wrappers are updating to include _trace_db_call function here.

But when Django access execute_wrappers here, it seems to be empty. Looking closely, I've found different object instances when the execute_wrappers are updated at first (<django.db.backends.sqlite3.base.DatabaseWrapper object at 0x7fa410750240>), and when it is accessed after (<django.db.backends.sqlite3.base.DatabaseWrapper object at 0x7fa4103a24a8>). Seems that Django instantiate multiple DatabaseWrapper objects, causing the problem.

If I change the initialization of execute_wrappers to process_request it works, but only log one query:

    def process_request(self, request):
        """Called on each request, before Django decides which view to execute.

        :type request: :class:`~django.http.request.HttpRequest`
        :param request: Django http request.
        """
        if django.VERSION >= (2,) and not _trace_db_call in connection.execute_wrappers:
            connection.execute_wrappers.append(_trace_db_call)

My environment:

Package                  Version  
------------------------ ---------
cachetools               3.1.1    
certifi                  2019.9.11
chardet                  3.0.4    
contextvars              2.4      
Django                   2.0      
google-api-core          1.14.3   
google-auth              1.7.0    
googleapis-common-protos 1.6.0    
idna                     2.8      
immutables               0.11     
mysql                    0.0.2    
mysql-connector          2.2.9    
mysqlclient              1.4.4    
opencensus               0.7.5    
opencensus-context       0.1.1    
opencensus-ext-dbapi     0.1.2    
opencensus-ext-django    0.7.2    
opencensus-ext-mysql     0.1.2    
opencensus-ext-zipkin    0.2.2    
pip                      19.3.1   
pkg-resources            0.0.0    
protobuf                 3.10.0   
pyasn1                   0.4.7    
pyasn1-modules           0.2.7    
PyMySQL                  0.9.3    
pytz                     2019.3   
requests                 2.22.0   
rsa                      4.0      
setuptools               39.0.1   
six                      1.12.0   
sqlparse                 0.3.0    
urllib3                  1.25.6

I'm trying to use it in the example included on plugin (https://github.com/census-instrumentation/opencensus-python/tree/master/contrib/opencensus-ext-django).

I just change settings to use SQLITE:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, '../db.sqlite3'),
    }
}

I'm doing something wrong?

[tpyo]

@jordanorc Just tested this and it seems you're right, although I'm only seeing this behaviour locally with runserver. Let's move this to a new issue though.