Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 30 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

### What is it?

PyTabular is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it. See [Documentation Here](https://curts0.github.io/PyTabular/)
PyTabular is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it. See [Documentation Here](https://curts0.github.io/PyTabular/). PyTabular is still considered alpha while I'm working on building out the proper tests and testing environments, so I can ensure some kind of stability in features.

### Getting Started

Expand All @@ -25,21 +25,41 @@ DAX Query

```python
model.Query(DAX_QUERY)

#Example Dax Query
#EVALUATE
#TOPN(100,'Table1')

# Returns a Pandas DataFrame
```

[Refresh Tables and Partitions](https://curts0.github.io/PyTabular/Tabular/#refresh)
See [Refresh Tables and Partitions](https://curts0.github.io/PyTabular/Tabular/#refresh).

```python
#Can be str(table name only), Table object, Partition object, or an iterable combination of the three.
#You have a few options when refreshing.
model.Refresh('Table Name')
tables_to_refresh = ['Table Name 1', 'Table Name 2', <Table Class>, <Partition Class>]
#Queue up the tables and partitions that you want to refresh.
model.Refresh(tables_to_refresh)
#NOTE if you monitor the logs you will notice a Trace is executed on the refreshes.

#or...
model.Refresh(['Table1','Table2','Table3'])

#or...
model.Refresh(<Table Class>)

#or...
model.Refresh(<Partition Class>)

#or...
model.Refresh({'Table Name':'Partition Name'})

#or any kind of weird combination like
model.Refresh([{<Table Class>:<Partition Class>,'Table Name':['Partition1','Partition2']},'Table Name','Table Name2'])

#Add Tracing=True for simple Traces tracking the refresh.
model.Refresh(['Table1','Table2'], Tracing=True)

```

Built In Dax Query Helpers
Built In Dax Query Helpers. In-case you want to run some quick queries similar to what vertipaq analyzer will do when getting row counts.
```python

#Query Every Column
Expand All @@ -57,14 +77,14 @@ Built In Dax Query Helpers
'''
```

Backup & Revert a Table in Memory
Backup & Revert a Table in Memory. USE WITH CAUTION, obviously not in PROD. I have been experimenting with this concept.
```python
model.Backup_Table('TableName') #This will backup the table with surround items (columns,measures,relationships,roles,hierarchies,etc.) and will add a suffix of '_backup'
#Make any changes to your original table and then revert or delete backup as necessary
model.Revert_Table('TableName') #This will essentially replace your original with _backup
```

Run BPA from TE2
Run BPA from TE2. Roadmap to make this more robust, and allow you to run all the command line interfaces that TE2 has to offer.
```python
TE2 = pytabular.TE2() #Feel free to input your TE2 File path or this will download for you.
BPA = pytabular.BPA() #Fee free to input your own BPA file or this will download for you from: https://raw.githubusercontent.com/microsoft/Analysis-Services/master/BestPracticeRules/BPARules.json
Expand Down
3 changes: 2 additions & 1 deletion pytabular/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,5 @@
from . logic_utils import pd_dataframe_to_m_expression, pandas_datatype_to_tabular_datatype
from . tabular_tracing import Base_Trace, Refresh_Trace
from . tabular_editor import Tabular_Editor
from . best_practice_analyzer import BPA
from . best_practice_analyzer import BPA
logging.debug(f'Import successful...')
19 changes: 9 additions & 10 deletions pytabular/pytabular.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,17 @@
logger = logging.getLogger('PyTabular')

logger.debug(f'Importing Microsoft.AnalysisServices.Tabular')
from Microsoft.AnalysisServices.Tabular import Server, Database, RefreshType, DataType, ConnectionDetails, ColumnType, MetadataPermission, Table, DataColumn, Partition, MPartitionSource, PartitionSourceType, Trace, TraceEvent, TraceEventHandler
from Microsoft.AnalysisServices.Tabular import Server, RefreshType, ColumnType, Table, DataColumn, Partition, MPartitionSource
logger.debug(f'Importing Microsoft.AnalysisServices.AdomdClient')
from Microsoft.AnalysisServices.AdomdClient import (AdomdCommand, AdomdConnection)
logger.debug(f'Importing Microsoft.AnalysisServices')
from Microsoft.AnalysisServices import UpdateOptions, TraceEventClass, TraceEventSubclass, TraceEventCollection, TraceColumn
from Microsoft.AnalysisServices import UpdateOptions

logger.debug('Importing Other Packages...')
from typing import Any, Dict, List, Optional, Union, Callable
from typing import Any, Dict, List, Union
from collections.abc import Iterable
from collections import namedtuple
import requests as r
import pandas as pd
import json
import os
import subprocess
import atexit
Expand Down Expand Up @@ -89,7 +87,7 @@ def Refresh(self,
str == 'Table_Name'
Table == Table Object
Partition == Partition Object
Dict[str, Any] == A way to specify a partition of group of partitions. For ex: {'Table_Name':'Partition1'} or {'Table_Name':['Partition1','Partition2']}. NOTE you can also change out the strings for partition or tables objects.
Dict[str, Any] == A way to specify a partition of group of partitions. For ex. {'Table_Name':'Partition1'} or {'Table_Name':['Partition1','Partition2']}. NOTE you can also change out the strings for partition or tables objects.
RefreshType (RefreshType, optional): See [RefreshType](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices.tabular.refreshtype?view=analysisservices-dotnet). Defaults to RefreshType.Full.
Tracing (bool, optional): Currently just some basic tracing to track refreshes. Defaults to False.

Expand Down Expand Up @@ -197,10 +195,11 @@ def property_changes(Property_Changes):
Added_Subtree_Roots = Model_Save_Results.Impact.AddedSubtreeRoots
Removed_Objects = Model_Save_Results.Impact.RemovedObjects
Removed_Subtree_Roots = Model_Save_Results.Impact.RemovedSubtreeRoots
Changes = namedtuple("Changes","Property_Changes Added_Objects Added_Subtree_Roots Removed_Objects Removed_Subtree_Roots")
[property_changes(Property_Changes), Added_Objects, Added_Subtree_Roots, Removed_Objects, Removed_Subtree_Roots]
Xmla_Results = Model_Save_Results.XmlaResults
Changes = namedtuple("Changes","Property_Changes Added_Objects Added_Subtree_Roots Removed_Objects Removed_Subtree_Roots Xmla_Results")
[property_changes(Property_Changes), Added_Objects, Added_Subtree_Roots, Removed_Objects, Removed_Subtree_Roots, Xmla_Results]
self.Reload_Model_Info()
return Changes(property_changes(Property_Changes), Added_Objects, Added_Subtree_Roots, Removed_Objects, Removed_Subtree_Roots)
return Changes(property_changes(Property_Changes), Added_Objects, Added_Subtree_Roots, Removed_Objects, Removed_Subtree_Roots, Xmla_Results)
def Backup_Table(self,table_str:str) -> bool:
'''USE WITH CAUTION, EXPERIMENTAL. Backs up table in memory, brings with it measures, columns, hierarchies, relationships, roles, etc.
It will add suffix '_backup' to all objects.
Expand Down Expand Up @@ -358,7 +357,7 @@ def Query(self,Query_Str:str) -> pd.DataFrame:
logger.debug(f'Converting Results into List...')
while Query.Read():
Results.append([Query.GetValue(index) for index in range(0,len(Column_Headers))])
logger.info(f'Data retrieved and closing query...')
logger.debug(f'Data retrieved and closing query...')
Query.Close()
logger.debug(f'Converting to Pandas DataFrame...')
df = pd.DataFrame(Results,columns=[value for _,value in Column_Headers])
Expand Down