Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 20 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

### What is it?

PyTabular (python-tabular in [pypi](https://pypi.org/project/python-tabular/)) is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it. See [Documentation Here](https://curts0.github.io/PyTabular/). PyTabular is still considered alpha while I'm working on building out the proper tests and testing environments, so I can ensure some kind of stability in features. Please send bugs my way! Preferably in the issues section in Github. I want to harden this project so many can use it easily. I currently have local pytest for python 3.6 to 3.11 and run those tests through a local AAS and Gen2 model.
PyTabular (python-tabular in [pypi](https://pypi.org/project/python-tabular/)) is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it. See [Documentation Here](https://curts0.github.io/PyTabular/). PyTabular is still considered alpha while I'm working on building out the proper tests and testing environments, so I can ensure some kind of stability in features. Please send bugs my way! Preferably in the issues section in Github. I want to harden this project so many can use it easily. I currently have local pytest for python 3.6 to 3.10 and run those tests through a local AAS and Gen2 model.

### Getting Started
See the [Pypi project](https://pypi.org/project/python-tabular/) for available version.
Expand All @@ -34,6 +34,10 @@ model.Query(DMV_QUERY) #returns pd.DataFrame()
#or...
SINGLE_VALUE_QUERY_EX = "EVALUATE {1}"
model.Query(SINGLE_VALUE_QUERY_EX) #returns 1

#or...
FILE_PATH = 'C:\\FILEPATHEXAMPLE\\file.dax' #or file.txt
model.Query(FILE_PATH) #Will return same logic as above, single values if possible else will return pd.DataFrame()
```

Refresh method to handle refreshes on your model. This is synchronous. Should be flexible enough to handle a variety of inputs. See [PyTabular Docs for Refreshing Tables and Partitions](https://curts0.github.io/PyTabular/Tabular/#refresh). Most basic way to refresh is input the table name string. The method will search for table and output exeption if unable to find it. For partitions you will need a key, value combination. Example, {'Table1':'Partition1'}. You can also take the key value pair and iterate through a group of partitions. Example, {'Table1':['Partition1','Partition2']}. Rather than providing a string, you can also input the actual class. See below for those examples, and you can acess them from the built in attributes self.Tables, self.Partitions or explore through the .Net classes yourself in self.Model.Tables.
Expand Down Expand Up @@ -62,7 +66,7 @@ model.Refresh(['Table1','Table2'], Tracing=True)

### Use Cases

##### If blank table, then refresh table.
#### If blank table, then refresh table.
This will use the function [Return_Zero_Row_Tables](https://curts0.github.io/PyTabular/Examples/#return_zero_row_tables) and the method [Refresh](https://curts0.github.io/PyTabular/Tabular/#refresh) from the Tabular class.
```python
import pytabular
Expand All @@ -72,7 +76,7 @@ if len(tables) > 0:
model.Refresh(tables, Tracing = True) #Add a trace in there for some fun.
```

##### Sneak in a refresh.
#### Sneak in a refresh.
This will use the method [Is_Process](https://curts0.github.io/PyTabular/Tabular/#is_process) and the method [Refresh](https://curts0.github.io/PyTabular/Tabular/#refresh) from the Tabular class. It will check the DMV to see if any jobs are currently running classified as processing.
```python
import pytabular
Expand All @@ -83,7 +87,7 @@ else:
model.Refresh(TABLES_OR_PARTITIONS_TO_REFRESH)
```

##### Show refresh times in model.
#### Show refresh times in model.
This will use the function [Table_Last_Refresh_Times](https://curts0.github.io/PyTabular/Examples/#table_last_refresh_times) and the method [Create_Table](https://curts0.github.io/PyTabular/Tabular/#create_table) from the Tabular class. It will search through the model for all tables and partitions and pull the 'RefreshedTime' property from it. It will return results into a pandas dataframe, which will then be converted into an M expression used for a new table.
```python
import pytabular
Expand All @@ -93,7 +97,7 @@ model.Create_Table(df, 'Refresh Times')
```


##### If BPA Violation, then revert deployment.
#### If BPA Violation, then revert deployment.
Uses a few things. First the [BPA Class](https://curts0.github.io/PyTabular/Best%20Practice%20Analyzer/#bpa), then the [TE2 Class](https://curts0.github.io/PyTabular/Tabular%20Editor%202/), and will finish with the [Analyze_BPA](https://curts0.github.io/PyTabular/Tabular/#analyze_bpa) method. Did not want to re-invent the wheel with the amazing work done with Tabular Editor and it's BPA capabilities.
```python
import pytabular
Expand All @@ -106,7 +110,7 @@ if len(results) > 0:
#Revert deployment here!
```

##### Backup & Revert a Table.
#### Backup & Revert a Table.
USE WITH CAUTION, obviously not in PROD. I have been experimenting with this concept. Made for selfish reasons. Will probably get removed and I'll keep in my own local version. But fun to work with. Uses two methods. [Backup_Table](https://curts0.github.io/PyTabular/Tabular/#backup_table) and [Revert_Table](https://curts0.github.io/PyTabular/Tabular/#revert_table)

```python
Expand All @@ -117,4 +121,14 @@ model.Backup_Table('TableName') #This will backup the table with surround items
#Make any changes to your original table and then revert or delete backup as necessary
#-----------#
model.Revert_Table('TableName') #This will essentially replace your original with _backup
```

#### Loop through and query Dax files
Let's say you have multiple dax queries you would like to store and run through as checks. The [Query](https://curts0.github.io/PyTabular/Tabular/#query) method on the Tabular class can also take file paths. Can really be any file type as it's just checking os.path.isfile(). But would suggest .dax or .txt. It will read the file that use that as the new Query_str argument.
```python
import pytabular
model = pytabular.Tabular(CONNECTION_STR)
LIST_OF_FILE_PATHS = ['C:\\FilePath\\file1.dax','C:\\FilePath\\file1.txt','C:\\FilePath\\file2.dax','C:\\FilePath\\file2.txt']
for file_path in LIST_OF_FILE_PATHS:
model.Query(file_path)
```
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "python_tabular"
version = "0.1.1"
version = "0.1.2"
authors = [
{ name="Curtis Stallings", email="curtisrstallings@gmail.com" },
]
Expand Down
9 changes: 2 additions & 7 deletions pytabular/pytabular.py
Original file line number Diff line number Diff line change
Expand Up @@ -372,26 +372,21 @@ def Query(self,Query_Str:str) -> Union[pd.DataFrame,str,int]:


try:
logger.debug(f'Attempting to Open Adomd Connection...')
logger.debug(f'Setting first initial Adomd Connection...')
self.AdomdConnection.Open()
logger.debug(f'Connected!')
except:
logger.debug(f'Connection skipped already connected...')
pass
logger.info(f'Querying Model...')
Query = AdomdCommand(Query_Str, self.AdomdConnection).ExecuteReader()
logger.debug(f'Determining Field Count...')
Column_Headers = [(index,Query.GetName(index)) for index in range(0,Query.FieldCount)]
Results = list()
logger.debug(f'Converting Results into List...')
while Query.Read():
Results.append([Query.GetValue(index) for index in range(0,len(Column_Headers))])
logger.debug(f'Data retrieved and closing query...')
Query.Close()
logger.debug(f'Converting to Pandas DataFrame...')
logger.debug(f'Data retrieved... reading...')
df = pd.DataFrame(Results,columns=[value for _,value in Column_Headers])
if len(df) == 1 and len(df.columns) == 1:
logging.debug(f'Returning single value...')
return df.iloc[0][df.columns[0]]
return df
def Query_Every_Column(self,query_function:str='COUNTROWS(VALUES(_))') -> pd.DataFrame:
Expand Down
2 changes: 2 additions & 0 deletions test/dfvaltest.dax
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
EVALUATE
{ ( 1, 2 ), ( 3, 4 ) }
2 changes: 2 additions & 0 deletions test/singlevaltest.dax
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
EVALUATE
{ 1 }
16 changes: 14 additions & 2 deletions test/test_tabular.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@
testing_parameters = [(aas),(gen2)]
testingtable = 'PyTestTable'

@pytest.mark.parametrize("model",testing_parameters)
def test_sanity_check(model):
assert 1 == 1

@pytest.mark.parametrize("model",testing_parameters)
def test_connection(model):
'''
Expand All @@ -22,12 +26,20 @@ def test_database(model):
assert isinstance(model.Database,Database)

@pytest.mark.parametrize("model",testing_parameters)
def test_query(model):
def test_basic_query(model):
int_result = model.Query('EVALUATE {1}')
text_result = model.Query('EVALUATE {"Hello World"}')
assert int_result == 1 and text_result == 'Hello World'


@pytest.mark.parametrize("model",testing_parameters)
def test_file_query(model):
singlevaltest = l.SINGLEVALTESTPATH
dfvaltest = l.DFVALTESTPATH
dfreplication = pd.DataFrame({'[Value1]':(1,3),'[Value2]':(2,4)})
assert model.Query(singlevaltest) == 1 and model.Query(dfvaltest).equals(dfreplication)


def remove_testing_table(model):
table_check = [table for table in model.Model.Tables.GetEnumerator() if testingtable in table.Name]
for table in table_check:
Expand Down Expand Up @@ -65,4 +77,4 @@ def test_table_removal(model):
def test_bpa(model):
te2 = pytabular.Tabular_Editor().EXE
bpa = pytabular.BPA().Location
assert model.Analyze_BPA(te2,bpa)
assert isinstance(model.Analyze_BPA(te2,bpa), list)