skfolio / skfolio Goto Github PK
View Code? Open in Web Editor NEWPython library for portfolio optimization built on top of scikit-learn
Home Page: https://skfolio.org
License: BSD 3-Clause "New" or "Revised" License
Python library for portfolio optimization built on top of scikit-learn
Home Page: https://skfolio.org
License: BSD 3-Clause "New" or "Revised" License
Is your feature request related to a problem? Please describe.
The name prior_estimator and PriorModel could be confusing for some users as it's not clear what model the prior refers to.
Describe the solution you'd like
A solution would be to rename prior_estimator by distribution_estimator and PriorModel() byDistribution
Describe alternatives you've considered
Alternative could be dist_estimator, Dist, DistModel
Additional context
This will impact the API of all optimization estimators by renaming the parameter prior_estimator.
Describe the bug
In all metrics the 255 business days standard is used.
On average US based calendar has 252 business days in a year.
While I understand this depends on the country and on the specific year, most annualization metrics should at least be clear about it.
Expected behavior
Annualization should be based on typically 252 business days for US based user.
Very nice package, stringently designed!
I was wondering whether you were thinking about sktime
integration?
sktime
has native support fo hierarchical data, pairwise distance transformers for time series, etc.skpro
.scikit-base
, which is similar to your "import BaseEstimator
" but imo more architecturally stringent. Especially if you want unified interface tests that go beyond sklearn
check_estimator
.Opening an issue since I'm not quite sure what the best way is to get in touch.
make it clear about linux ubuntu installation ?
in
https://cvxopt.org/install/index.html#installing-a-pre-built-package
With multiperiod walk-forward portfolio, we are able to grab the compositions over the multiple rebalancing periods. It would be great to have a date-matched composition i.e., portfolio weights, even if the weights repeat for multiple days, I can think of a dataframe days x assets, and then of course the NaN return dates will have zero weight for the respective asset. I have around 5 more suggestions to make, I will open a new issue for each. This is what the composition looks like currently.
maybe .portfolio_weights
, .daily_weights
something like that could distinguish from above.
Describe the bug
Since Ruff v0.2.0 the configuration of ruff using the pyproject.toml file changed.
To Reproduce
any ruff command will echo a warning encouraging the user to update the ruff configuration.
ruff check
Expected behavior
No warning should be displayed.
Versions
0.1.2
It seems the MeanRisk class ignores the input value of risk_free_rate and keeps it fixed at 0 no matter what the user inputs.
You can run the example below and notice that the risk free rate is printed as 0 even if it is input as 0.02/252.
You can run the code below.
from yfinance import download
DATA = download(["TSLA","SBUX","CAKE"], start="2019-5-1", end="2024-5-1")
ADJUSTED = DATA["Adj Close"]
from skfolio.preprocessing import prices_to_returns
RETURNS = prices_to_returns(ADJUSTED)
from skfolio.optimization import MeanRisk, ObjectiveFunction
MODEL = MeanRisk(objective_function=ObjectiveFunction.MAXIMIZE_RATIO , risk_free_rate=0.02/252)
P = MODEL.fit_predict(RETURNS)
print( P.risk_free_rate)
Expected behavior
The printed risk free rate should not equal 0.
Funds may contain multiple sectors. For example, the A fund may invest 40% to banking, 30% tech, 30% estate and the B fund may invest 20% to banking and 80% to industry. In group-and-linear-constraints page, asset groups can be assigned.
Let weights assigned to be these groups
such as {A: {banking: 0.4, tech: 0.3, estate: 0.3}
, so a constraint to banking counts A
as 0.4
and B
as 0.2
.
It would be very useful for a portfolio composed of ETFs.
Thanks for the great-looking package!
I have hopefully a small request. Currently you have:
Line 16 in a1e79d3
Is it possible to change this to 3.9
to reduce the restrictiveness?
Sklearn doesn't actually specify a python version in their project.toml. And I don't see a need to restrict from the current dependencies:
dependencies = [
"numpy>=1.23.4",
"scipy>=1.8.0",
"pandas>=1.4.1",
"cvxpy>=1.4.1",
"scikit-learn>=1.3.2",
"joblib>=1.3.2",
"plotly>=5.15.0"
]
Currently the rolling measure plot only works for portfolios and not for populations. This could be a helpful addition. Expected behaviour.
population.plot_rolling_measure(measure=RatioMeasure.SHARPE_RATIO)
Then if 4 difference portfolios in population, they would all be plotted
from skfolio import Portfolio
X = [
[0.003, -0.001],
[-0.001, 0.002],
[0.0015, 0.004],
]
weights = [0.6, 0.4]
portfolio = Portfolio(X=X, weights=weights)
print(portfolio.returns)
>>> array([0.0014, 0.0002, 0.0025])
Starting from here, should there not be a weight that allows for 2D input? So instead of a static portfolio, a dynamic one.
For example, If I rebalance every month,
X = [
[0.003, -0.001],
...new months
[-0.001, 0.002],
[0.0015, 0.004],
....
]
weights = [
[0.90, 0.10],
..new month
[0.45, 0.55],
[0.45, 0.55],
....
]
portfolio = Portfolio(X=X, weights=weights)
Is it possible to have an example where the precision matrix obtained with GraphicalLassoCV is employed?
GINI_MEAN_DIFFERENCE Risk Measure for MeanRisk model not working. I have tried with SCIP and CLARABEL solvers. Looks like I am missing something. Appreciate any help. Thanks!
model = MeanRisk(
risk_measure=RiskMeasure.GINI_MEAN_DIFFERENCE,
objective_function=ObjectiveFunction.MAXIMIZE_RATIO,
portfolio_params=dict(name="Max Sharpe"),
solver='CLARABEL'
)
model.fit(X)
model.weights_-->
Is your feature request related to a problem? Please describe.
Effective number of assets is a metric related to portfolio concentration.
Describe the solution you'd like
Having a new metric taking a weights array and returning the inverse of the sum of squared weights.
wikipedia page
Describe the bug
When integrating 2 portfolios into the Population
class that result from an MeanRisk()
with a third portfolio that was created with the Portfolio
class, when I try to use the plot_cumulative_returns()
method, only 1 portfolio is shown.
To Reproduce
X = pd.DataFrame(np.random.rand(365,3), columns=["SPY", "GOVT", "QQQ"],
index=pd.date_range(start='2018-01-01', end='2018-12-31'))
model = MeanRisk(
risk_measure=RiskMeasure.VARIANCE,
objective_function=ObjectiveFunction.MAXIMIZE_RATIO,
portfolio_params=dict(name="Max Sharpe"),
min_weights={"SPY": 0.3, "GOVT": 0.1},
max_weights={"SPY": 0.30001, "GOVT": 0.10001}
)
benchmark_inv = InverseVolatility(portfolio_params=dict(name="Inverse Vol"))
benchmark = Portfolio(X["SPY"].to_numpy().reshape(-1, 1), weights=[1.0], name="OnlySPY")
benchmark_inv.fit(X)
model.fit(X)
pred_model = model.predict(X)
pred_bench = benchmark_inv.predict(X)
fig = Population([pred_model, benchmark, pred_bench]).plot_cumulative_returns()
show(fig)
Expected behavior
The image should show the three portfolio cumulative returns.
Versions
import skfolio; print(skfolio.__version__)
0.1.2
Describe the bug
IndexError is raised when computing the maximum amount of clusters when the amount of columns in the returns dataframe is low.
│ /Users/xxx/dev/xxx/venv/lib/python3.11/site-packages/skfolio/optimization/cluster/hierar │
│ chical/_hrp.py:326 in fit │
│ │
│ 325 │ │ │
│ ❱ 326 │ │ self.hierarchical_clustering_estimator_.fit(distance) │
│ 327 │
│ │
│ /Users/xxx/dev/xxx/venv/lib/python3.11/site-packages/skfolio/cluster/_hierarchical.py:19 │
│ 6 in fit │
│ │
│ 195 │ │ if max_clusters is None: │
│ ❱ 196 │ │ │ max_clusters = compute_optimal_n_clusters( │
│ 197 │ │ │ │ distance=X, │
│ │
│ /Users/xxx/dev/xxx/venv/lib/python3.11/site-packages/skfolio/utils/stats.py:454 in │
│ compute_optimal_n_clusters │
│ │
│ 453 │ for k in range(max_clusters): │
│ ❱ 454 │ │ level = cut_tree[:, n - k - 1] │
│ 455 │ │ cluster_density = [] │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: index -4 is out of bounds for axis 1 with size 3
To Reproduce
import pandas as pd
from skfolio.optimization import HierarchicalRiskParity
from skfolio.moments import DenoiseCovariance
from skfolio.distance import CovarianceDistance
from skfolio import RiskMeasure
df = pd.read_parquet("X_before.parquet")
model = HierarchicalRiskParity(risk_measure=RiskMeasure.MEAN_ABSOLUTE_DEVIATION,portfolio_params={"name": "HRP-MAD-Ward-DenoisedPearson"},distance_estimator=CovarianceDistance(covariance_estimator=DenoiseCovariance()))
model.fit(df)
Expected behavior
I believe this comes from the code here which defines max clusters at a minimum as 8, whereas the amount of columns could be less than 8.
Additional context
If HRP shouldn't be used for less than 8 return columns this could be exposed and allowed to be checked for by the consumer to avoid catching the IndexError
. Otherwise I would suggest taking off the 8 max and changing it to the length of the columns array.
Versions
0.2.1
Scikit-learn released Metadata Routing in v1.4.
Integrating this feature into skfolio
will make it cleaner and easier to use extrageneous datasets into meta estimators.
For example, creating a Covariance estimator that uses the assets' implied volatility time series in addition to the assets' returns time series. Then using such estimator in meta estimators such as Optimization estimators, pipelines and cross validators.
Let's see type hints as documentation for the IDE :)
Let me suggest some improvements to some type hints encountered throughout the code.
I will upload this issue alongside my PR.
The goal of this issue is not to discuss design choices and their implications on the hints but to help with minor suggestions.
Type issues:
any
the python built-on instead of Any
the type.Describe the bug
When running the example file ‘plot_2_herc_cdar.py’, the results of the drawing are not displayed. The same problem occurs when running my own HERC analyses.Thanks!
Is your feature request related to a problem? Please describe.
Add equation (3) from the Exposure Stacking article https://ssrn.com/abstract=4709317 as final_estimator in StackingOptimization.
Describe the solution you'd like
Adding the inner sum from equation (3) as a final_estimator objective in StackingOptimization. To adapt the method to the case without parameter uncertainty, I suggest that the data is split into K folds (L in the article) on which optimal portfolio exposures are computed for each estimator and then combined using the Exposure Stacking objective.
If the functionality is at some point used in combination with resampled parameter uncertainty as in the article, the number of folds parameter L = 2 seems to be a good default value, while it will be interesting to experiment with that one as well. Especially if multiple portfolio estimators are applied to the parameter samples.
Describe alternatives you've considered
Alternatives are the current implementation with portfolio optimization objectives that are implemented elegantly in this package already.
Additional context
Exposure Stacking is a method to stack the results of different portfolio optimizations that is independent of any investment risk measure and, hence, a good candidate when portfolios have been optimized using different portfolio optimization objectives. For the resampled parameter uncertainty case, it alleviates the need for adding additional constraints on risk and return to avoid significant unintended drift from the average risk and return of the optimized portfolios.
Describe the bug
prices_to_returns
promises to take a dataframe of prices and convert them to returns. In the event I have a dataframe with returns with different starting times (such as VTI
and BITI
) this function will cut rows to the start of BITI
.
To Reproduce
from skfolio.preprocessing import prices_to_returns
import yfinance as yf
import pandas as pd
STOCKS = ["VTI", "BITI"]
return_dfs = []
for ticker in STOCKS:
ticker_yahoo = yf.Ticker(ticker)
df = ticker_yahoo.history(period="1d", start="2010-01-01")
df.index = df.index.date
return_dfs.append(df[["Close"]].rename(columns={"Close": ticker}))
return_df = pd.concat(return_dfs, axis=1).sort_index().fillna(0.0)
prices_to_returns(return_df)
Expected behavior
Returns for BITI
before 2022-06-21
are 0.0. Instead it cuts off the dataframe.
Additional context
This occurs due to the dropna. I believe this should use fillna
with 0.0 after the pct_change
call to mark the previous periods are 0.0 for returns. I would be more than happy to make this change just want to be sure this is considered the right move by the maintainers.
Versions
0.2.1
Wow, super cool library & docs.
I had not seen sphinx-gallery docs before, they look nice. It looks like the gallery includes a jupyterlite environment where you can run notebooks like this; I haven't seen that before either.
Given the docs are hosted on GitHub pages for now, I guess you probably can't install and run a kernel in that environment. It seemed that it couldn't quite find the notebooks, for example, https://skfolio.org/lite/lab/?path=auto_examples/1_mean_risk/plot_1_maximum_sharpe_ratio.ipynb
, but maybe that can be fixed in conf.py via these docs - maybe the notebook can still be displayed nicely in jupyterlite, just not run, which would probably still be cool.
Either way, maybe you'd want to temporarily remove allusions to the jupyterlite pages from your docs, e.g. plot_1_maximum_sharpe_ratio.html#L399 & plot_1_maximum_sharpe_ratio.html#L836 with a script or something, or maybe can just be done in sphinx-gallery build.
https://skfolio.org/user_guide/datasets.html
The user is requesting the addition of documentation on how to customize datasets. Currently, the native datasets provided only include a portion of stocks from the S&P 500 and NASDAQ. In order to construct a portfolio of ETFs or stocks from other countries' markets, there is a need for custom datasets.
Provide information on how to create and customize datasets, ideally with relevant code examples.
Describe the bug
The title of the CombinatorialPurgedCV.plot_train_test_folds() is probably misleading.
Otherwise it happens that the portfolio is fitted with very small training sets and tested on very long ones.
To Reproduce
from skfolio.model_selection import CombinatorialPurgedCV
CombinatorialPurgedCV().plot_train_test_folds()
Expected behavior
I believe train is 1 and test is 0.
Versions
0.1.1
I also did an analysis based on this encoding and the results are similar:
Is your feature request related to a problem? Please describe.
FYI ruff now includes a formatter faster than black and with very similar behavior.
Describe the solution you'd like
Drop black in favor of ruff to slim skfolio's optional-dependencies.
Additional context
a format step could be added to the tests job to warn if code is not properly formatted.
As of now, the CV is valuable, but to bring it closer to asset management as opposed to data science the rebalancing ordinarily doesn't occur in a number of days, but perhaps on the 15th of every month or every 4th Friday.
Expected behaviour:
##polars nomenclature
self.cv = WalkForwardDate(period=252, every="4w", offset="-3d") ## This would run every 4th Friday.
##asset management nomenclature
self.cv = WalkForwardDate(lookback=252, rebalance="4w", offset="-3d") ## This would run every 4th Friday.
Here I am taking inspiration from the group_by_dynamic
from polars.
df_signal_long.group_by_dynamic(
index_column=date_col,
every=every,
period=f"{lookback}{every[-1]}" if isinstance(lookback, int) else lookback,
by=entity_col,
closed="left",
label="left",
offset="-3d", # Offset by 3 days to end on Friday
include_boundaries=False
)
We currently have MultiPeriodPortfolio.composition
which is helpful, it would be great if we could do the same for MultiPeriodPortfolio.contribution
, which would allow us again to have something similar to composition over the different test sets,
Or as recommended in #69 (comment) a date type Contribution that rolls forward.
Securities often have different lenghts, how would you deal with this? Thanks for the package.
Describe the bug
The last two lines "Portfolios number" and "Avg nb of assets per portfolio" report the wrong dimension in the resulting dataframe of the Population.summary()
when objects resulting from a cross_val_predict
are used to initialize the Population
object.
To Reproduce
from skfolio.model_selection import WalkForward
from skfolio import RiskMeasure, RatioMeasure
from skfolio.optimization import MeanRisk, HierarchicalRiskParity, EqualWeighted
from skfolio.population import Population
from skfolio.model_selection import cross_val_predict
estimators = {
"minvol": MeanRisk(risk_measure=RiskMeasure.VARIANCE, transaction_costs=0),
"mad": MeanRisk(
risk_measure=RiskMeasure.MEAN_ABSOLUTE_DEVIATION, transaction_costs=0
),
"ew": EqualWeighted(),
"hrp": HierarchicalRiskParity(risk_measure=RiskMeasure.VARIANCE,transaction_costs=0),
}
cv_preds = {}
for name, estimator in estimators.items():
cv_preds[name] = cross_val_predict(
estimator=estimator,
X=train_returns,
cv=WalkForward(test_size=252, train_size=9 * 252, expend_train=False),
n_jobs=1,
)
cv_pop = Population(list(cv_preds.values()))
cv_pop.summary()
Expected behavior
The "Avg nb of assets per portfolio" could be the average of the effective number of assets (like described in PR #17).
The portfolios number is not clear to me though.
Versions
0.0.9
Describe the bug
skfolio
calls pandas
plot
to create its graphs. It sets the backend in places like this when skfolio
is imported. However this may be changed by other operators, so when it comes time to plot issues like this can arise:
│ /Users/xxx/dev/xxx/venv/lib/python3.11/site-packages/skfolio/population/_population.py:6 │
│ 20 in plot_cumulative_returns │
│ │
│ 619 │ │ fig = df.plot() │
│ ❱ 620 │ │ fig.update_layout( │
│ 621 │ │ │ title=title, │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'Axes' object has no attribute 'update_layout'
In this case the backend has been changed to matplotlib
by other libraries, while skfolio
expects the backend to be plotly
.
To Reproduce
from skfolio.datasets import load_sp500_dataset
from skfolio.preprocessing import prices_to_returns
from skfolio.optimization import EqualWeighted
from skfolio import Population
import pandas as pd
prices = load_sp500_dataset()
X = prices_to_returns(prices)
model = EqualWeighted()
model.fit(X)
pop = Population([model.predict(X)])
pd.options.plotting.backend = "matplotlib"
pop.plot_cumulative_returns()
Expected behavior
No crash, during the plot_cumulative_returns
call the backend is set to plotly
and then set back again at the end of the call.
Additional context
I would be more than happy to make this change, just want to check that maintainers agree with the diagnosis.
Versions
0.2.1
Are there any reasons we are setting the import dependencies are set so high? For example, when installing skfolio
alongside other libraries, it often leads to installation failures. Starting from Python 3.10, the following dependency specifications should work fine, adhering to minimal dependency versioning or minimal sufficient versioning. We can probably just test these to see if they work:
python = "^3.10"
numpy = ">=1.20"
pandas = ">=1.0"
plotly = ">=5.0"
scipy = ">=1.0"
scikit-learn = ">=1.0"
Can Skfolio add support for python3.9? Because some commonly used quantization packages like qlib support only up to python3.9.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.