Skip to content

Prog models features/sim result with pandas #526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 27 commits into
base: dev
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
5a26738
incorporating Dataframes instead of container
mstraut Mar 13, 2023
a0e5deb
updated containers.py
mstraut Apr 12, 2023
126f3c7
Update containers.py
mstraut Apr 12, 2023
e1ad3ff
Update containers.py
mstraut Apr 12, 2023
11bb02e
Update containers.py
mstraut Apr 12, 2023
092d19e
Delete battery_circuit_df.py
mstraut Apr 12, 2023
8ca1c4c
Update containers.py
mstraut Apr 12, 2023
56a2f3b
Delete prognostics_model_df.py
mstraut Apr 12, 2023
713d3aa
Merge remote-tracking branch 'origin/prog_models_features/container_r…
mstraut Apr 13, 2023
8210f04
working on issue with containers. see log.
mstraut Apr 13, 2023
2e7cce0
testing and correcting containers for full incorporation of pandas
mstraut Apr 15, 2023
1307c79
testing and correcting containers for full incorporation of pandas - …
mstraut Apr 17, 2023
783c974
testing and correcting containers for full incorporation of pandas - …
mstraut Apr 18, 2023
1de2960
container changes and updates
mstraut Apr 18, 2023
d2d2d44
container changes and updates
mstraut Apr 18, 2023
d24710d
Finished containers __init__
mstraut Apr 20, 2023
cd4408b
container update 4/26/2023
mstraut Apr 27, 2023
b91ffca
container update 4/27/2023
mstraut Apr 27, 2023
174ccf4
container completion 5/1/2023
mstraut May 1, 2023
9380d6c
updating sim result,
mstraut May 2, 2023
03b9eb0
updating sim result,
mstraut May 3, 2023
4f9b572
updating sim result, LazySimResult
mstraut May 3, 2023
0e452ca
completed sim result, LazySimResult. double checking
mstraut May 3, 2023
9086eb9
completed sim result, LazySimResult. double checking
mstraut May 5, 2023
ca1bcfe
tests complete
mstraut May 6, 2023
8e3f1ba
Merge branch 'dev' into prog_models_features/sim_result_with_pandas
mstraut May 6, 2023
db0c4b3
Revert "Merge branch 'dev' into prog_models_features/sim_result_with_…
mstraut May 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/pr-messages.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ jobs:
pullRequestOpened: >
Thank you for opening this PR. Each PR into dev requires a code review. For the code review, look at the following:

- [ ] Reviewer (someone other than author) should look for bugs, efficiency, readability, testing, and coverage in examples (if relevant).
- [ ] Reviewer should look for bugs, efficiency, readability, testing, and coverage in examples (if relevant).

- [ ] Ensure that each PR adding a new feature should include a test verifying that feature.

- [ ] All tests must be passing.

- [ ] All errors from static analysis must be resolved.

- [ ] Review the test coverage reports (if there is a change) - will be added as comment on PR if there is a change
Expand Down
67 changes: 28 additions & 39 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -110,42 +110,31 @@ jobs:
- name: Run copyright check
run: |
python scripts/test_copyright.py
coverage:
timeout-minutes: 30
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ['3.10']
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e .
pip install coverage
pip install notebook
pip install testbook
pip install requests
pip install importlib_metadata
- name: Run coverage
run: |
coverage run -m tests.test_base_models
coverage run -a -m tests.test_battery
coverage run -a -m tests.test_centrifugal_pump
coverage run -a -m tests.test_composite
coverage run -a -m tests.test_datasets
coverage run -a -m tests.test_dict_like_matrix_wrapper
coverage run -a -m tests.test_direct
coverage run -a -m tests.test_ensemble
coverage run -a -m tests.test_linear_model
coverage run -a -m tests.test_pneumatic_valve
coverage run -a -m tests.test_powertrain
coverage run -a -m tests.test_serialization
coverage run -a -m tests.test_sim_result
coverage xml -i
- name: "Upload coverage to Codecov"
uses: codecov/codecov-action@v3
# coverage:
# timeout-minutes: 30
# runs-on: ubuntu-latest
# strategy:
# matrix:
# python-version: ['3.9']
# steps:
# - uses: actions/checkout@v3
# - name: Set up Python ${{ matrix.python-version }}
# uses: actions/setup-python@v4
# with:
# python-version: ${{ matrix.python-version }}
# - name: Install dependencies
# run: |
# python -m pip install --upgrade pip
# python -m pip install -e .
# pip install coverage
# pip install notebook
# pip install testbook
# pip install requests
# - name: Run coverage
# run: |
# coverage run -m tests
# coverage xml
# - name: "Upload coverage to Codecov"
# uses: codecov/codecov-action@v3
# with:
# fail_ci_if_error: true
5 changes: 0 additions & 5 deletions codecov.yml

This file was deleted.

6 changes: 0 additions & 6 deletions examples/param_est.py
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,5 @@ def run_example():

# Sure enough- parameter estimation determined that the thrower's height wasn't 20 m, instead was closer to 1.9m, a much more reasonable height!

# Note: You can also adjust the metric that is used to estimate parameters.
# This is done by setting the "error_method" argument.
# e.g., m.estimate_params([(times, inputs, outputs)], keys, dt=0.01, error_method='MAX_E')
# Default is Mean Squared Error (MSE)
# See calc_error method for list of options.

if __name__=='__main__':
run_example()
11 changes: 5 additions & 6 deletions src/prog_models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,10 @@
# National Aeronautics and Space Administration. All Rights Reserved.

# PrognosticsModel must be first, since the others build on this
from prog_models.prognostics_model import PrognosticsModel
from prog_models.ensemble_model import EnsembleModel
from prog_models.composite_model import CompositeModel
from prog_models.linear_model import LinearModel
from prog_models.models.thrown_object import LinearThrownObject
from prog_models.exceptions import ProgModelException, ProgModelInputException, ProgModelTypeError
from .prognostics_model import PrognosticsModel
from .ensemble_model import EnsembleModel
from .composite_model import CompositeModel
from .linear_model import LinearModel
from .exceptions import ProgModelException, ProgModelInputException, ProgModelTypeError

__version__ = '1.5.0.pre'
42 changes: 23 additions & 19 deletions src/prog_models/composite_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

from collections.abc import Iterable

from prog_models import PrognosticsModel
from . import PrognosticsModel

DIVIDER = '.'

Expand Down Expand Up @@ -32,7 +32,8 @@ class CompositeModel(PrognosticsModel):
outputs (list[str]):
Model outputs in format "model_name.output_name". Must be subset of all outputs from models. If not provided, all outputs will be included.
"""
def __init__(self, models, connections = [], **kwargs):

def __init__(self, models, connections=[], **kwargs):
# General Input Validation
if not isinstance(models, Iterable):
raise ValueError('The models argument must be a list')
Expand All @@ -54,8 +55,9 @@ def __init__(self, models, connections = [], **kwargs):
# Handle models
for m in models:
if isinstance(m, Iterable):
if len(m) != 2:
raise ValueError('Each model tuple must be of the form (name: str, model). For example ("Batt1", BatteryElectroChem())')
if len(m) != 2:
raise ValueError(
'Each model tuple must be of the form (name: str, model). For example ("Batt1", BatteryElectroChem())')
if not isinstance(m[0], str):
raise ValueError('The first element of each model tuple must be a string')
if not isinstance(m[1], PrognosticsModel):
Expand All @@ -73,15 +75,16 @@ def __init__(self, models, connections = [], **kwargs):
self.model_names.add(m[0])
kwargs['models'].append(m)
else:
raise ValueError(f'Each model must be a PrognosticsModel or tuple (name: str, PrognosticsModel), was {type(m)}')
raise ValueError(
f'Each model must be a PrognosticsModel or tuple (name: str, PrognosticsModel), was {type(m)}')

for (name, m) in kwargs['models']:
self.inputs |= set([name + DIVIDER + u for u in m.inputs])
self.states |= set([name + DIVIDER + x for x in m.states])
self.outputs |= set([name + DIVIDER + z for z in m.outputs])
self.events |= set([name + DIVIDER + e for e in m.events])
self.performance_metric_keys |= set([name + DIVIDER + p for p in m.performance_metric_keys])

# Handle outputs
if 'outputs' in kwargs:
if isinstance(kwargs['outputs'], str):
Expand All @@ -91,11 +94,11 @@ def __init__(self, models, connections = [], **kwargs):
if not set(kwargs['outputs']).issubset(self.outputs):
raise ValueError('The outputs of the composite model must be a subset of the outputs of the models')
self.outputs = kwargs['outputs']

# Handle Connections
kwargs['connections'] = []
self.__to_input_connections = {m_name : [] for m_name in self.model_names}
self.__to_state_connections = {m_name : [] for m_name in self.model_names}
self.__to_input_connections = {m_name: [] for m_name in self.model_names}
self.__to_state_connections = {m_name: [] for m_name in self.model_names}

for connection in connections:
# Input validation
Expand All @@ -107,13 +110,14 @@ def __init__(self, models, connections = [], **kwargs):
in_key, out_key = connection
# Validation
if out_key not in self.inputs:
raise ValueError(f'The output key, {out_key}, must be an input to one of the composite models. Options include {self.inputs}')
raise ValueError(
f'The output key, {out_key}, must be an input to one of the composite models. Options include {self.inputs}')

# Remove the out_key from inputs
# These no longer are an input to the composite model
# as they are now satisfied internally
self.inputs.remove(out_key)

# Split the keys into parts (model, key_part)
(in_model, in_key_part) = in_key.split('.')
(out_model, out_key_part) = out_key.split('.')
Expand All @@ -125,7 +129,7 @@ def __init__(self, models, connections = [], **kwargs):
raise ValueError('The input model must be one of the models in the composite model')
if out_model not in self.model_names:
raise ValueError('The output model must be one of the models in the composite model')

# Add to connections
if in_key in self.states:
self.__to_input_connections[out_model].append((in_key, out_key_part))
Expand All @@ -138,11 +142,11 @@ def __init__(self, models, connections = [], **kwargs):
self.states.add(in_key)
else:
raise ValueError('The input key must be an output or state of one of the composite models')

# Finish initialization
super().__init__(**kwargs)

def initialize(self, u = {}, z = {}):
def initialize(self, u={}, z={}):
if u is None:
u = {}
if z is None:
Expand All @@ -156,7 +160,7 @@ def initialize(self, u = {}, z = {}):
x_i = m.initialize(u_i, z_i)
for key, value in x_i.items():
x_0[name + '.' + key] = value

# Process connections
# This initializes the states that are connected to outputs
for (in_key_part, in_key) in self.__to_state_connections[name]:
Expand All @@ -165,7 +169,7 @@ def initialize(self, u = {}, z = {}):
else: # Missing from z, so estimate using initial state
z_ii = m.output(x_i)
x_0[in_key] = z_ii.get(in_key_part, None)

return self.StateContainer(x_0)

def next_state(self, x, u, dt):
Expand All @@ -177,17 +181,17 @@ def next_state(self, x, u, dt):
for (in_key, out_key_part) in self.__to_input_connections[name]:
u_i[out_key_part] = x[in_key]
u_i = m.InputContainer(u_i)

# Prepare state
x_i = m.StateContainer({key: x[name + '.' + key] for key in m.states})

# Propogate state
# Propagate state
x_next_i = m.next_state(x_i, u_i, dt)

# Save to super state
for key, value in x_next_i.items():
x[name + '.' + key] = value

# Process connections
# This updates the states that are connected to outputs
if len(self.__to_state_connections[name]) > 0:
Expand Down
21 changes: 4 additions & 17 deletions src/prog_models/ensemble_model.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
# Copyright © 2021 United States Government as represented by the Administrator of the
# National Aeronautics and Space Administration. All Rights Reserved.

from collections.abc import Sequence
import numpy as np

from prog_models import PrognosticsModel
from . import PrognosticsModel


class EnsembleModel(PrognosticsModel):
Expand All @@ -20,7 +19,7 @@ class EnsembleModel(PrognosticsModel):
See example :download:`examples.ensemble <../../../../prog_models/examples/ensemble.py>`

Args:
models (list[PrognosticsModel]): List of at least 2 models that form the ensemble
models (list): List of models that form the ensemble

Keyword Arguments:
aggregation_method (function): Function that aggregates the outputs of the models in the ensemble. Default is np.mean
Expand All @@ -31,14 +30,6 @@ class EnsembleModel(PrognosticsModel):
}

def __init__(self, models, **kwargs):
if not isinstance(models, Sequence):
raise TypeError(f'EnsembleModel must be initialized with a list of models, got {type(models)}')
if len(models) < 2:
raise ValueError('EnsembleModel requires at least two models')
for i, m in enumerate(models):
if not isinstance(m, PrognosticsModel):
raise TypeError(f'EnsembleModel requires all models to be PrognosticsModel instances. models[{i}] was {type(m)}')

inputs = set()
states = set()
outputs = set()
Expand All @@ -56,12 +47,8 @@ def __init__(self, models, **kwargs):
super().__init__(**kwargs)
self.parameters['models'] = models

def initialize(self, u=None, z=None):
xs = [
m.initialize(
m.InputContainer(u) if u is not None else None,
m.OutputContainer(z) if z is not None else None
) for m in self.parameters['models']]
def initialize(self, u, z=None):
xs = [m.initialize(m.InputContainer(u), m.OutputContainer(z) if z is not None else None) for m in self.parameters['models']]
x0 = {}
for x in xs:
for key in x.keys():
Expand Down
Loading