Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 7 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

### What is it?

PyTabular is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it.
PyTabular is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it. See [Documentation Here](https://curts0.github.io/PyTabular/)

### Getting Started

Expand All @@ -28,17 +28,17 @@ DAX Query
# Returns a Pandas DataFrame
```

Refresh Tables and Partitions
[Refresh Tables and Partitions](https://curts0.github.io/PyTabular/Tabular/#refresh)

```python
#filter down the collection to what you want to refresh
tables_to_refresh = [table for table in model.Tables if table.Name in ['Table1','Table2','Table3']]

#Can be str(table name only), Table object, Partition object, or an iterable combination of the three.
model.Refresh('Table Name')
tables_to_refresh = ['Table Name 1', 'Table Name 2', <Table Class>, <Partition Class>]
#Queue up the tables and partitions that you want to refresh.
model.Refresh(tables_to_refresh)

#Once you are ready, update to execute the refresh
model.Update()
model.SaveChanges()
```

Built In Dax Query Helpers
Expand Down Expand Up @@ -71,6 +71,4 @@ Run BPA from TE2
TE2 = pytabular.TE2() #Feel free to input your TE2 File path or this will download for you.
BPA = pytabular.BPA() #Fee free to input your own BPA file or this will download for you from: https://raw.githubusercontent.com/microsoft/Analysis-Services/master/BestPracticeRules/BPARules.json
model.Analyze_BPA(TE2.EXE_Path,BPA.Location) #This will output a list of BPA violations...
```

I'm working on converting everything to the google docstring format and using those to populate the documentation. I still have some formatting issues to work through, but that should help with understanding what this package can do.
```
Binary file removed dist/python_tabular-0.0.20.tar.gz
Binary file not shown.
Binary file not shown.
Binary file added dist/python_tabular-0.0.30.tar.gz
Binary file not shown.
8 changes: 3 additions & 5 deletions docs/BPA.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


## BPA
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L374)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L393)
```python
BPA(
rules_location: str = 'https: //raw.githubusercontent.com/microsoft/Analysis-Services/master/BestPracticeRules/BPARules.json'
Expand All @@ -11,7 +11,5 @@ BPA(


---
Best Practice Analyzer Class
Can provide Url, Json File Path, or Python List.
If nothing is provided it will default to Microsofts Analysis Services report with BPA Rules.
https://raw.githubusercontent.com/microsoft/Analysis-Services/master/BestPracticeRules/BPARules.json
_summary_

27 changes: 27 additions & 0 deletions docs/Examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,30 @@ Example to add to model model.Create_Table(p.Table_Last_Refresh_Times(model),'Re

* **DataFrame** : pd dataframe with the RefreshedTime property: https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices.tabular.partition.refreshedtime?view=analysisservices-dotnet#microsoft-analysisservices-tabular-partition-refreshedtime
If group_partition == True and the table has multiple partitions, then df.groupby(by["tables"]).max()

----


### BPA_Violations_To_DF
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/basic_checks.py\#L50)
```python
.BPA_Violations_To_DF(
model: pytabular.Tabular, te2: str, bpa: str
)
```

---
Runs BPA Analyzer from TE2 and outputs result into a DF.


**Args**

* **model** (pytabular.Tabular) : Tabular Model Class
* **te2** (str) : TE2 Exe File Path (Can use TE2().EXE_path)
* **bpa** (str) : BPA File Location (Can use BPA().Location)


**Returns**

* **DataFrame** : Super simple right now. Just splits into two columns.. The object in violation and the rule.

9 changes: 4 additions & 5 deletions docs/TE2.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


## TE2
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L399)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L417)
```python
TE2(
TE_Location = 'https: //github.com/TabularEditor/TabularEditor/releases/download/2.16.7/TabularEditor.Portable.zip'
Expand All @@ -11,7 +11,6 @@ TE2(


---
TE2 Class, to use any built TabularEditor Command Line Scripts
https://docs.tabulareditor.com/te2/Command-line-Options.html
#https://github.com/TabularEditor/TabularEditor/releases/download/2.16.7/TabularEditor.Portable.zip
#https://cdn.tabulareditor.com/files/TabularEditor.2.16.7.zip
TE2 Class, to use any built TabularEditor Command Line Scripts
[TE2 Command Line Example](https://docs.tabulareditor.com/te2/Command-line-Options.html)
[TE2 Download](https://github.com/TabularEditor/TabularEditor/releases/download/2.16.7/TabularEditor.Portable.zip)
49 changes: 31 additions & 18 deletions docs/Tabular.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@


## Tabular
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L20)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L21)
```python
Tabular(
CONNECTION_STR: str
Expand All @@ -23,8 +23,23 @@ Tabular Class to perform operations:[Microsoft.AnalysisServices.Tabular](https:/
**Methods:**


### .Reload_Model_Info
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L53)
```python
.Reload_Model_Info()
```

---
Runs on __init__ iterates through details, can be called after any model changes. Called in SaveChanges()


**Returns**

* **bool** : True if successful


### .Disconnect
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L55)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L64)
```python
.Disconnect()
```
Expand All @@ -39,27 +54,25 @@ Disconnects from Model


### .Refresh
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L70)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L79)
```python
.Refresh(
iterable_items: List, RefreshType = RefreshType.Full
Object: Union[str, Table, Partition, Iterable], RefreshType = RefreshType.Full
)
```

---
Input iterable Collections for the function to run through.
It will add the collection items into a Refresh Request.
To execute refresh run through Update()
Input Object(s) to be refreshed in the tabular model. Combine with .SaveChanges() to actually run the refresh on the model.


**Args**

* **iterable_items** (List) : Must be refreshable Tabular objects.
* **RefreshType** (_type_, optional) : _description_. Defaults to RefreshType.Full.
* **Object** (Union[str,Table,Partition,Iterable]) : Can be str(table name only), Table object, Partition object, or an iterable combination of the three.
* **RefreshType** (_type_, optional) : [RefreshType](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices.tabular.refreshtype?view=analysisservices-dotnet). Defaults to RefreshType.Full.


### .Update
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L82)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L100)
```python
.Update(
UpdateOptions: UpdateOptions = UpdateOptions.ExpandFull
Expand All @@ -81,7 +94,7 @@ To execute refresh run through Update()


### .SaveChanges
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L93)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L111)
```python
.SaveChanges()
```
Expand All @@ -96,7 +109,7 @@ Just a simple wrapper to call self.Model.SaveChanges()
bool:

### .Backup_Table
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L102)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L121)
```python
.Backup_Table(
table_str: str
Expand All @@ -120,7 +133,7 @@ Refresh is performed from source during backup.


### .Revert_Table
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L169)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L188)
```python
.Revert_Table(
table_str: str
Expand Down Expand Up @@ -148,7 +161,7 @@ Example scenario ->


### .Query
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L237)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L256)
```python
.Query(
Query_Str: str
Expand All @@ -170,7 +183,7 @@ Executes Query on Model and Returns Results in Pandas DataFrame


### .Query_Every_Column
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L267)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L286)
```python
.Query_Every_Column(
query_function: str = 'COUNTROWS(VALUES(_))'
Expand All @@ -193,7 +206,7 @@ This will dynamically create a query to pull all columns from the model and run


### .Query_Every_Table
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L289)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L308)
```python
.Query_Every_Table(
query_function: str = 'COUNTROWS(_)'
Expand All @@ -216,7 +229,7 @@ It will replace the _ with the table to run.


### .Analyze_BPA
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L309)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L328)
```python
.Analyze_BPA(
Tabular_Editor_Exe: str, Best_Practice_Analyzer: str
Expand All @@ -241,7 +254,7 @@ https://docs.tabulareditor.com/te2/Command-line-Options.html


### .Create_Table
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L333)
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/pytabular.py\#L352)
```python
.Create_Table(
df: pd.DataFrame, table_name: str
Expand Down
42 changes: 42 additions & 0 deletions docs/Utils.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,45 @@ WiP takes dataframe columns and gets respective tabular column datatype. ([NumP

* **Dict** : EX {'col1': <Microsoft.AnalysisServices.Tabular.DataType object at 0x0000023BFFBC9700>, 'col2': <Microsoft.AnalysisServices.Tabular.DataType object at 0x0000023BFFBC8840>, 'col3': <Microsoft.AnalysisServices.Tabular.DataType object at 0x0000023BFFBC9800>}


----


### pd_dataframe_to_m_expression
[source](https://github.com/Curts0/PyTabular\blob\master\pytabular/logic_utils.py\#L75)
```python
.pd_dataframe_to_m_expression(
df: pd.DataFrame
)
```

---
This will take a pandas dataframe and convert to an m expression
For example this DF:
col1 col2
0 1 3
1 2 4

|
|
V

Will convert to this expression string:
let
Source=#table({"col1","col2"},
{
{"1","3"},{"2","4"}
})
in
Source


**Args**

* **df** (pd.DataFrame) : Pandas DataFrame


**Returns**

* **str** : Currently only returning string values in your tabular model.

16 changes: 7 additions & 9 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

### What is it?

PyTabular is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it.
PyTabular is a python package that allows for programmatic execution on your tabular models! This is possible thanks to [Pythonnet](https://pythonnet.github.io/) and Microsoft's [.Net APIs on Azure Analysis Services](https://docs.microsoft.com/en-us/dotnet/api/microsoft.analysisservices?view=analysisservices-dotnet). The package should have the dll files included when you import it. See [Documentation Here](https://curts0.github.io/PyTabular/)

### Getting Started

Expand All @@ -28,17 +28,17 @@ DAX Query
# Returns a Pandas DataFrame
```

Refresh Tables and Partitions
[Refresh Tables and Partitions](https://curts0.github.io/PyTabular/Tabular/#refresh)

```python
#filter down the collection to what you want to refresh
tables_to_refresh = [table for table in model.Tables if table.Name in ['Table1','Table2','Table3']]

#Can be str(table name only), Table object, Partition object, or an iterable combination of the three.
model.Refresh('Table Name')
tables_to_refresh = ['Table Name 1', 'Table Name 2', <Table Class>, <Partition Class>]
#Queue up the tables and partitions that you want to refresh.
model.Refresh(tables_to_refresh)

#Once you are ready, update to execute the refresh
model.Update()
model.SaveChanges()
```

Built In Dax Query Helpers
Expand Down Expand Up @@ -71,6 +71,4 @@ Run BPA from TE2
TE2 = pytabular.TE2() #Feel free to input your TE2 File path or this will download for you.
BPA = pytabular.BPA() #Fee free to input your own BPA file or this will download for you from: https://raw.githubusercontent.com/microsoft/Analysis-Services/master/BestPracticeRules/BPARules.json
model.Analyze_BPA(TE2.EXE_Path,BPA.Location) #This will output a list of BPA violations...
```

I'm working on converting everything to the google docstring format and using those to populate the documentation. I still have some formatting issues to work through, but that should help with understanding what this package can do.
```
4 changes: 3 additions & 1 deletion mkgendocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,10 @@ pages:
functions:
- Return_Zero_Row_Tables
- Table_Last_Refresh_Times
- BPA_Violations_To_DF
- page: "Utils.md"
source: 'pytabular/logic_utils.py'
functions:
- ticks_to_datetime
- pandas_datatype_to_tabular_datatype
- pandas_datatype_to_tabular_datatype
- pd_dataframe_to_m_expression
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "python_tabular"
version = "0.0.20"
version = "0.0.30"
authors = [
{ name="Curtis Stallings", email="curtisrstallings@gmail.com" },
]
Expand Down
10 changes: 10 additions & 0 deletions pytabular/basic_checks.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,16 @@ def Table_Last_Refresh_Times(model:pytabular.Tabular, group_partition:bool = Tru
return df

def BPA_Violations_To_DF(model:pytabular.Tabular,te2:str, bpa:str) -> pd.DataFrame:
'''Runs BPA Analyzer from TE2 and outputs result into a DF.

Args:
model (pytabular.Tabular): Tabular Model Class
te2 (str): TE2 Exe File Path (Can use TE2().EXE_path)
bpa (str): BPA File Location (Can use BPA().Location)

Returns:
pd.DataFrame: Super simple right now. Just splits into two columns.. The object in violation and the rule.
'''
results = model.Analyze_BPA(te2,bpa)
data = [rule.replace(' violates rule ','^').replace('\"','').split('^') for rule in results]
columns = ["Object","Violation"]
Expand Down
Loading