kearnz / autoimpute Goto Github PK

View Code? Open in Web Editor NEW

237.0 237.0 19.0 1.72 MB

Python package for Imputation Methods

License: MIT License

Python 100.00%

autoimpute's Introduction

👋👋

I'm Joe. I also go by kearnz.
Software engineer at rocketship.vc
How to reach me: [email protected] 📬
Also, Co-author, developer, and maintainer of Autoimpute. Check it out below ⬇️

autoimpute's People

Contributors

Stargazers

Watchers

Forkers

shabarka bigdatamatta mlataibrahim gilacnuvo gjdv blaxe05 priyatamnayak satishjasthi csetraynor demirtonchev prashant-rahul kushaldev75 tanglespace mitchell-xiyunfeng virtualphoton gtesei iq-scm pinto-katende-jonathan hbi0

autoimpute's Issues

Bug Using Bayesian Linear Regression and Bayesian Logistic Regression in MiceImputer

There's a bug in how the MiceImputer implements bayesian methods. Note that both bayesian binary and bayesian least squares suffer from this issue. Essentially, pymc3, the underlying package we leverage for building bayesian models, does not allow you to redefine an existing deterministic variable. autoimpute tries to do that when it iterates through imputations. I will work on a fix for this bug this weekend when I'm tackling another issue.

Here's where that pops up in autoimpute. Both the MultipleImputer and the MiceImputer create n SingleImputer instances under the hood. In the MultipleImputer, each of those n_i instances iterates k=1 time. So if you use a bayesian method, the bayesian model variables are created 1 time for each n instances. Perfectly valid. But for the MiceImputer, each n_1 instances of the SingleImputer iterate k=5 (by default) times. So each instance tries to recreate bayesian variables k times, and that throws an error.

Adding convergence criterion and monitoring to MiceImputer

Problem

The MiceImputer accepts a keyword k for the number of iterations. There is no easy way of determining the optimal k depending on the data at hand.

Currently, I am fixing the imputation with a seed while increasing k to monitor the convergence of the imputed variables. This is code-wise ok but computationally very inefficient.

Possible Solutions

I see two solutions which could both be implemented:

Adding a convergence/stopping criterion based on the elements of the difference matrix between iterations. This would be a keyword of the MiceImputer constructor
Caching the imputation results between iterations and creating a plot_convergence function which displays the change of imputed variables over the iterations, as in Figures 7-10 in Buuren+ 2011, or here.

Alternative to new feature

A less user-friendly but more exhaustive solution would be to optionally return all iteration results for each imputation run, not just the final one. Assuming I'm instantiating MiceImputer with k=10, n=5, and return_list=True, I would then be getting a list with 5 elements (just as now), where each element is itself a list with 10 elements, each element being the data matrix at various imputation steps.

I have not seen this being addressed in any branch of the code. I'm also not very familiar with MICE, so maybe there is a better way of determining k that I'm not aware of. If you think this feature could be valuable, I'd be happy to contribute code in any part you see fit.

Priors for Bayesian Methods

Default value of priors for mu and sd in numerical bayesian methods for imputation should likely be from the observed series for which we are trying to predict. Right now they are hard-coded at 0 and 10 respectively. Eventually, the user will have the option to pass these values, but until then, we still likely need better defaults.

Bad priors can lead to slow convergence for MCMC and even numerical overflow issues with pymc3 when realistic values are not within the prior interval.

cannot split between train and test set, scientific integrity not possible

In research, scientific integrity plays a very important part. One can publish very good papers by playing tricks between train and test set in order to get good results, but such results can never be applied in real life, because those tricks simply does not work in real-life applications.

Your framework provides fit, transform and fit_transform, the architecture is very sound. However, for every strategy, there should be an option specifying whether to take statistics from the training set or test set. For example, for 'mean imputation', should it be the train-set mean or test-set mean. These discussions give a very good explanation on that: 1 2

The above code shows one major issue of your framework. If you have 3 datasets, and you train imputation model on dataset 1 (using any of the strategies), and then, whether you test on dataset 2 or dataset 2&3 combined, you will get different results. Somehow, some of the statistics still comes from the test set. This will result in systematic inconsistency. Thanks!

timeseries integration for MissingnessPredictor class

Right now, class deals with categorical and numerical datatypes. Not sure how timeseries data handled, and need to account for it if project expands to time series analysis

correlation and covariance in patterns

Right now, the patterns file does not support methods that return correlation and covariance matrices. Ideally, these methods would be included in patterns, where users can explore data using a variety of (already implemented) methods

Error : X has 6 features per sample; expecting 7

trying to use autoimpute on kaggle favorita data :
store_nbr item_nbr unit_sales onpromotion store_nbr_item_nbr
0 17 581742 8.0 NaN 17_581742
1 14 902200 7.0 NaN 14_902200
2 17 581742 7.0 NaN 17_581742
3 14 902200 1.0 NaN 14_902200
4 17 581742 3.0 NaN 17_581742

when doing :

    imp = SingleImputer() # or     imp = MiceImputer(return_list=True)
    imp = MiceImputer(return_list=True)
    imp.fit_transform(temp)

reviving the above error .
when removing "store_nbr_item_nbr" every thing is ok

multipleimputer fit_transform gives error ( it is probably not finding indexes for sampling)

Hi. Ihave been trying to run the package but keep geting the following error,. To the best of my knowledge it has to be relates to not finding enough indexes to sample.

ValueError Traceback (most recent call last)
in
1 auto_imputer=MultipleImputer(return_list=True)
----> 2 df_imp=auto_imputer.fit_transform(df_total)

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in fit_transform(self, X, y)
229 def fit_transform(self, X, y=None):
230 """Convenience method to fit then transform the same dataset."""
--> 231 return self.fit(X, y).transform(X)

~/anaconda3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
59 err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
60 raise TypeError(err)
---> 61 return func(d, *args, **kwargs)
62 return wrapper
63

~/anaconda3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
124
125 # return func if no missingness violations detected, then return wrap
--> 126 return func(d, *args, **kwargs)
127 return wrapper
128

~/anaconda3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
171 err = f"All values missing in column(s) {nc}. Should be removed."
172 raise ValueError(err)
--> 173 return func(d, *args, **kwargs)
174 return wrapper
175

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in transform(self, X)
224 for i in self.statistics_.items())
225 if self.return_list:
--> 226 imputed = list(imputed)
227 return imputed
228

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in (.0)
222 # sequential only for now
223 imputed = ((i[0], i[1].transform(X))
--> 224 for i in self.statistics_.items())
225 if self.return_list:
226 imputed = list(imputed)

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/single_imputer.py in transform(self, X)
251 for col in x_m:
252 d = DefaultUnivarImputer()
--> 253 d_imps = d.fit_impute(x_[col], None)
254 x_null = x_[col][x_[col].isnull()].index
255 x_.loc[x_null, col] = d_imps

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/series/default.py in fit_impute(self, X, y)
217 def fit_impute(self, X, y):
218 """Convenience method to perform fit and imputation in one go."""
--> 219 return self.fit(X, y).impute(X)
220
221 class DefaultUnivarImputer(DefaultBaseImputer):

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/series/default.py in impute(self, X)
276 def impute(self, X):
277 """Defer transform to the DefaultBaseImputer."""
--> 278 X_ = super().impute(X)
279 return X_
280

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/series/default.py in impute(self, X)
212 # ensure that param is not none, which indicates time series column
213 if imp:
--> 214 X_ = imp.impute(X)
215 return X_
216

~/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/series/mode.py in impute(self, X)
118 imp = imp[0]
119 else:
--> 120 samples = np.random.choice(imp, len(ind))
121 imp = pd.Series(samples, index=ind).values
122

mtrand.pyx in mtrand.RandomState.choice()

ValueError: 'a' cannot be empty unless no samples are taken

finish tests for patterns file

Need to include:

tests for proper returned values from check_data_structure
tests for check_dimensions
tests for md_pairs, md_patterns
tests for inbound, outbound, influx, outflux, proportions, and flux

Docstrings to follow PEP257 style guide

Need to rewrite function signatures so that they are compliant with PEP257. Right now, some function docstrings not informative or properly written.

MissingnessClassifier prediction bug when classifier drops columns during fit

*** ValueError: labels [] not contained in axis ***

Above error happens when:

Columns dropped during fit b/c they are all NA
fit_predict or fit_predict_proba being used

Need to detect removed column during the predict stage.

Option to use Initialized Imputed DataFrame (not just observed data) for Imputation

Right now, the MultipleImputer does not use the initialized imputed dataframe, but rather selects only complete rows in the imputation. This occurs when each SingleImputer within a MultipleImputer calls the _get_observed method. While this is a common practice when strict MCAR or MAR conditions are met, using observed only can lead to selection bias when these conditions break down.

@gjvd provided this note (pg 10). In our implementation, the user should have the option to use observed only or use initialized DataFrame. This can be implemented as a formal argument with default value set.

multiple_imputer not doing real multiple imputation

While debugging some weird behavior (huge imputed values), I found that the multiple_imputer skips various important steps in MI as defined by Rubin and as implemented in the MICE R-package by Stef van Buuren.
MI should do the following:

repeat n times:
  identify missingness in dataframe
  initialize an imputed dataframe by inserting e.g., mean values per column where data is missing
  while not stable (or for set k number of iterations):
    for each column with missingness:
      create a single imputer using the current column as output and the other columns as input to the model
      update the imputed dataframe with imputed values where originally data was missing

Instead, multiple_imputer.py does not iteratively improve the imputed values, but rather runs the while-not-stable loop only once. Also, it does not use the initialized imputed dataframe, but rather selects only complete rows in the imputation (single_imputer calls _get_observed). This results in selection bias in the imputation.

Finally, the default imputation strategies default to regular regression methods rather than their Bayesian variants. As a consequence, the results of n imputations are all the same, e.g.,:

imp = MultipleImputer(return_list=True, n=3)
all(train_imp[0][1] == train_imp[1][1])   # gives True

If I find some time in near future, I might be able to help fixing this in the implementation.

Can't use date or time columns as predictors for imputation

Right now, time columns must be fully observed, and time columns are not imputed. That being said, they can still get through as predictors in multivariate predictive imputation methods.

Unfortunately, Autoimpute checks that multivariate predictive imputation methods have features that are numerical only. As a result, when a time-based column is passed to this check (specifically _not_num_matrix), the following error below is thrown:

si = SingleImputer()
si.fit_transform(df_with_ts_column)

TypeError: default predictive not appropriate for Matrix with non-numerical columns.

The error itself is not the bug. It says that the default predictive imputation strategy is not appropriate when trying to fit a matrix with non-numerical columns. Therefore, _not_num_matrix does it's job as expected. The issue is the fact that a time-series column can get to this stage without:

Being removed from the predictor set
Being encoded and stored in predictor set as encoded

Ideally, we want to incorporate time-series columns as predictors by encoding them (option 2), although the work to do so is non-trivial. We have to decide whether or not we will go through with dirty-work (encoding, scaling, etc.) or force that responsibility on the user.

Pooled Parameter Inference for sklearn in addition to statsmodels

Right now, the analysis models (MiLinearRegression and MiLogisticRegression) pool parameters via Rubin's rules if and only if statsmodels library is used. The reason for this is because sklearn does not support statistical analysis methods (standard error, etc.) out of the box, while statsmodels does.

The simple solution is to just use statsmodels, but we want to support sklearn end-to-end since this project is based on its design and Autoimpute uses sklearn everywhere else.

The current solution is to offer statsmodels and sklearn, but pool parameters if statsmodels is the library specified by the user (it's the argument default in class instantiation). If sklearn is the library, pooled coefficients are returned and prediction is possible, but no further inference.

The long term solution is to extend what we do for statsmodels to sklearn. To do so, we'll need to calculate the standard error of coefficients by hand. While not necessarily trivial to do, the resulting standard error should be easy to plug into the package. All other pooled metrics either directly start with standard error or are derived from metrics that need standard error, so for now, that should be all we need to calculate by hand.

checking same vector length within multi-dimensional arrays

nested lists and tuples must have the same vector length for each component vector. Right now, numpy silently converts to an array of lists, which ends up being 1D. The error output is confusing, and a check must be in place to make sure that nested lists have the same component vector length.

kwargs for Predictive model initiation, fit, and transformation

Right now, the PredictiveImputer allows for linear regression and logistic regression. It utilizes the implementations in sklearn. That being said, a user cannot pass customize the predictive models.

Customizing the models should be an option, although its implementation is a bit challenging. One must specify kwargs either by model type (i.e. kwargs for linear, separate kwargs for logistic) or by column designation (i.e. salary column gets salary_kwargs, age column gets age_kwargs). I guess this should happen in the init method for PredictiveImputer.

More complicated is distinguishing fit kwargs as well, although supporting kwargs on init may be enough to start.

dimension mismatch error on fit_transform for MultipleImputer

I'm trying to impute the missing values using autoImpute package for the titanic test data set using the python autoimpute package. However, the module is throwing a dimension mismatch error on the test data set.

kaggle titanic test data

import pandas as pd
from autoimpute.imputations import MultipleImputer

## titanic database test csv
X_test = pd.read_csv('test.csv') ## response is missing
X_test = X_test.drop(labels=['PassengerId','Name','Ticket','Cabin'], axis=1)

test_imp = MultipleImputer(seed = 1, return_list=True,
                     strategy={"Age": "pmm", "Fare": "pmm"},
                     imp_kwgs={"pmm": {"normalize": True, "n_jobs" : -1}},
                     )
print(X_test.isnull().sum())
X_test = test_imp.fit_transform(X_test)[0][1]

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-22-322f4b0b8702> in <module>
     13                      )
     14 print(X_test.isnull().sum())
---> 15 X_test = test_imp.fit_transform(X_test)[0][1]

~/intelpython3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in fit_transform(self, X, y)
    229     def fit_transform(self, X, y=None):
    230         """Convenience method to fit then transform the same dataset."""
--> 231         return self.fit(X, y).transform(X)

~/intelpython3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

~/intelpython3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

~/intelpython3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    171             err = f"All values missing in column(s) {nc}. Should be removed."
    172             raise ValueError(err)
--> 173         return func(d, *args, **kwargs)
    174     return wrapper
    175 

~/intelpython3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in transform(self, X)
    224                    for i in self.statistics_.items())
    225         if self.return_list:
--> 226             imputed = list(imputed)
    227         return imputed
    228 

~/intelpython3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in <genexpr>(.0)
    222         # sequential only for now
    223         imputed = ((i[0], i[1].transform(X))
--> 224                    for i in self.statistics_.items())
    225         if self.return_list:
    226             imputed = list(imputed)

~/intelpython3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

~/intelpython3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

~/intelpython3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    171             err = f"All values missing in column(s) {nc}. Should be removed."
    172             raise ValueError(err)
--> 173         return func(d, *args, **kwargs)
    174     return wrapper
    175 

~/intelpython3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/single_imputer.py in transform(self, X)
    261 
    262             # perform imputation given the specified imputer and value for x_
--> 263             X.loc[imp_ix, column] = imputer.impute(x_)
    264         return X
    265 

~/intelpython3/lib/python3.7/site-packages/autoimpute/imputations/series/pmm.py in impute(self, X)
    190         print(X.T.shape)
    191         print(X.T)
--> 192         y_pred_bayes = alpha_bayes + beta_bayes.dot(X.T)
    193         n_ = self.neighbors
    194         if X.columns.size == 1:

ValueError: shapes (7,) and (4,1) not aligned: 7 (dim 0) != 4 (dim 0)

However, the train dataset goes through the imputation without a problem. And the problem persists on the latest Anaconda distribution as well.

Also, the test dataset imputation works fine without the "Fare" part of the strategy. However, as a second step, the imputation with a new instance of the MultipleImputer with "Fare" and "completed" dataset from the first step, still produce same error.

Thanks a lot.

Predictor Placeholder Imputation within PredictiveImputer

PredictiveImputer supports machine learning models for imputation. Right now, it does so by fitting a model (linear, logistic) on observed X and y (note: the values for X and y depend on what the user specifies as predictors when creating an instance of the class). From the fit on observed, we derive coefficients which we then use for imputation during the transform step.

During the transform step, we predict values for Y_mis using the covariates X_mis, or the values for the predictors where Y is missing. That being said, nothing guarantees that X_mis is fully observed. If covariates have missing values, we must impute them with something so that we can generate predictions for each y in Y_mis. Right now, we impute the predictors in the transform step with the default methods from the SingleImputer.

There are two questions with this we may want to address:

Should the SingleImputer be customizable as well?
Once a column is imputed, should imputed values be used in subsequent predictions?

The first question is just a matter of writing extra code and validation. The second is a bit more theoretical. It's more necessary for multiple imputation, where we use something like the visit sequence or the random selection process.

Bug in MissingnessClassifier -> TypeError: data type 'k' not understood

The following works:

data = [{'a': 1, 'b': 2},{'a': 5, 'b': 10, 'c': 20}]
df = pd.DataFrame(data)
print(df)

   a   b     c
0  1   2   NaN
1  5  10  20.0

MissingnessClassifier().fit(df)

MissingnessClassifier(classifier=XGBClassifier(base_score=None, booster=None,
                                               colsample_bylevel=None,
                                               colsample_bynode=None,
                                               colsample_bytree=None,
                                               gamma=None, gpu_id=None,
                                               importance_type='gain',
                                               interaction_constraints=None,
                                               learning_rate=None,
                                               max_delta_step=None,
                                               max_depth=None,
                                               min_child_weight=None,
                                               missing=nan,
                                               monotone_constraints=None,
                                               n_estimators=100, n_jobs=None,
                                               num_parallel_tree=None,
                                               random_state=None,
                                               reg_alpha=None, reg_lambda=None,
                                               scale_pos_weight=None,
                                               subsample=None, tree_method=None,
                                               validate_parameters=None,
                                               verbosity=None))

However, when I name column 'a' -> 'k':

data_test = [{'k': 1, 'b': 2},{'k': 5, 'b': 10, 'c': 20}]
df = pd.DataFrame(data_test)
print(df)

   k   b     c
0  1   2   NaN
1  5  10  20.0

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-125-8fdb2eb9e304> in <module>
----> 1 MissingnessClassifier().fit(df)

C:\ProgramData\Miniconda3\envs\project_prorail\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

C:\ProgramData\Miniconda3\envs\project_prorail\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

C:\ProgramData\Miniconda3\envs\project_prorail\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    171             err = f"All values missing in column(s) {nc}. Should be removed."
    172             raise ValueError(err)
--> 173         return func(d, *args, **kwargs)
    174     return wrapper
    175 

C:\ProgramData\Miniconda3\envs\project_prorail\lib\site-packages\autoimpute\imputations\mis_classifier.py in fit(self, X, **kwargs)
    135         for column in self.data_mi:
    136             # only fit non time-based columns...
--> 137             if not np.issubdtype(column, np.datetime64):
    138                 y = self.data_mi[column]
    139                 preds = self._preds[column]

C:\ProgramData\Miniconda3\envs\project_prorail\lib\site-packages\numpy\core\numerictypes.py in issubdtype(arg1, arg2)
    386     """
    387     if not issubclass_(arg1, generic):
--> 388         arg1 = dtype(arg1).type
    389     if not issubclass_(arg2, generic):
    390         arg2 = dtype(arg2).type

TypeError: data type 'k' not understood

Does anyone have a clue why this would happen and how to fix?

python version 3.8.3

MICE Imputer does not implement the MICE algorithm

The series imputers used by the MICE Imputer, still call _get_observed (

autoimpute/autoimpute/imputations/dataframe/single_imputer.py

Line 185 in d1a4c39

x_, y_ = _get_observed(xs, ys)

)

However, the point of the MICE algorithm is to not use only the complete dataset, but to use the previous round's imputations as placeholders in the variables that are not currently being imputed (if I am imputing column x i want to use imputed placeholders in columns y, and z before doing the predictive imputing of column x)

For the first round, missing values are replaced by simple placeholders (mean/mode imputation) - additionally the order in which the columns are imputed at each round should be customizable

In the current implementation, the algorihm simply runs the multiple imputer K times, and the result is equal to the K-1th run, regardless of the results of the previous runs

See for reference: Azur, Melissa J., et al. "Multiple imputation by chained equations: what is it and how does it work?." International journal of methods in psychiatric research 20.1 (2011): 40-49.

Analysis Models not working with Categorical Predictors

Analysis models cannot fit data when predictors or response are categorical. MultipleImputation is not affected. the MultipleImputation happens before fitting a linear or logistic model with sklearn or statsmodels.

That being said, both libraries then fit analysis models on each of m imputed datasets. The imputed datasets preserve the structure of the original data, imputing actual string values rather than encodings.

So when an analysis model faces the imputed dataset, it gives a ValueError:

ValueError: Pandas data cast to numpy dtype of object. Check input data with np.asarray(data).

This error tells us that neither statsmodels nor sklearn are happy with the categorical columns present in imputed datasets.

better categorical encoding integration & MissingnessClassifier Design

Right now, only one-hot encoding supported via pd.get_dummies(). Encoding should leverage the sklearn contrib for categorical encoding. Additionally, data_numeric and data_dummy do not need to be separated if copy of original data stored

issue with groupby in md_pattern method

Currently throws ValueError: Grouper and axis must be same length

Actively looking into issues with changes to helper functions to try to resolve.

Dummy Variable Trap when feature is completely observed

Right now, we one-hot encode categorical features. When one-hot encoding, we leave out a column that specifies missing. For example, a gender column with values M/F/NA would have 3 columns, but we drop the NA column to avoid perfect multicollinearity (as all information is contained within two of the three dummy columns).

There's an issue, however, when not all features have missing values. If a feature such as gender is simply M/F, then our one-hot method returns 2 columns, and one must be dropped before gender is used in analysis.

data type homogeneity within columns of arrays

Right now, checks only enforce that an arrays have an acceptable structure. That being said, the datatypes within each column of an array must be homogeneous. This is automatically enforced by pandas dataframes, but not by lists or tuples transformed into 2D arrays.

Need to enforce that columns within an array have the same datatype so that imputations by column don't have mixed types.

Error while installing

Trying to install it by
pip install autoimpute
ERROR: Could not find a version that satisfies the requirement autoimpute (from versions: none)
ERROR: No matching distribution found for autoimpute

Integration with Sklearn Pipeline

I wanted to integrate auto impute with Sklearn pipeline as seen here :

https://scikit-learn.org/stable/auto_examples/compose/plot_column_transformer_mixed_types.html

However, I get the error AttributeError: 'generator' object has no attribute 'size' when I try and substitute SimpleImputer(strategy='median') with MultipleImputer().

edit : from what I can understand Sklearn imputer.fit_transform(X) returns an array whereas auto impute returns a generator object, which can then be fed into a DataFrame .

Thanks

Different predictors between imputation and analysis models

Right now, the MiLinearRegression and MiLogisticRegression classes take a dataset to impute and perform analysis. The classes do not allow the end user to specify more predictors for the imputation phase than the analysis phase. The classes assume that the feature set for regression is the same feature set used for imputation. The feature set can be less (if the user specifies fewer predictors, but the feature set cannot contain more columns for imputation than it does for regression.

A future improvement would allow the user to separate imputation from analysis so that the imputation phase can make use of auxillary variables not specified for analysis models. We can proceed in two ways:

Introduce a new argument that allows separation between the imputation predictors and the model predictors. This would allow everything to happen within the class, although it would break from the standard approach in statsmodels and sklearn where the fit method applies to a specified feature set.
Allow the analysis model classes to act on a dataset that's already been multiply imputed. This would probably be more clear, although it would separate the code into two steps.

Build Multinomial Bayes for Default Classification

Right now, the default classification method is multinomial logistic regression. As a result, imputed values are the same between imputations. The default should be a bayesian multinomial classifier, but we have not implemented one yet. There is a binary bayesian classifier, but not a multinomial one.

Once a multinomial version is built, we will set it as the default.

Make it easier to add new imputation methods and use unsupported ones

SingleImputer and MultipleImputer should support easier addition of new imputers for a given session. Additionally, both imputers should take any class that implements the ISeriesImputer as a strategy. This would make it easier for people to develop their own imputers and use them as they please.

numeric only and categorical only missingness

MissingnessIndicator handles mixed categorical and numeric. Error occurs if only numeric and only categorical. Need to make the class work for numeric only and categorical only

What is MiLinearRegression doing to my dataframe?

I'm following the guide about half way down this page: https://pypi.org/project/autoimpute/, in the section about MiLinearRegression, and I can't get past the .fit method.

I'll paste my exact code here.

Splitting the data:

training_features, testing_features, training_target, testing_target = train_test_split(data, labels)

And just a quick check to make sure the target variable is not in our training data:

'Churned' in training_features.columns

False

False. Ok, so I set up MiLinearRegression

from autoimpute.analysis import MiLinearRegression

simple_lm = MiLinearRegression()

# fit the model on each multiply imputed dataset and pool parameters
simple_lm.fit(training_features, training_target)




ValueError                                Traceback (most recent call last)
<ipython-input-135-d90445f8e8ef> in <module>
      1 # fit the model on each multiply imputed dataset and pool parameters
----> 2 simple_lm.fit(training_features, training_target)
      3 
      4 # get summary of fit, which includes pooled parameters under Rubin's rules
      5 # also provides diagnostics related to analysis after multiple imputation

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    171             err = f"All values missing in column(s) {nc}. Should be removed."
    172             raise ValueError(err)
--> 173         return func(d, *args, **kwargs)
    174     return wrapper
    175 

~\Anaconda3\lib\site-packages\autoimpute\analysis\linear_regressor.py in fit(self, X, y)
     88         # then fit the analysis models on each of the imputed datasets
     89         self.models_ = self._apply_models_to_mi_data(
---> 90             self.linear_models, X, y
     91         )
     92 

~\Anaconda3\lib\site-packages\autoimpute\analysis\base_regressor.py in _apply_models_to_mi_data(self, model_dict, X, y)
    248         model_type = model_dict["type"]
    249         regressor = model_dict[self.model_lib]
--> 250         mi_data = self._fit_strategy_validator(X, y)
    251         models = {}
    252 

~\Anaconda3\lib\site-packages\autoimpute\analysis\base_regressor.py in _fit_strategy_validator(self, X, y)
    205 
    206         # return the multiply imputed datasets
--> 207         return self.mi.fit_transform(X)
    208 
    209     def _fit_model(self, model_type, regressor, X, y):

~\Anaconda3\lib\site-packages\autoimpute\imputations\dataframe\multiple_imputer.py in fit_transform(self, X, y)
    229     def fit_transform(self, X, y=None):
    230         """Convenience method to fit then transform the same dataset."""
--> 231         return self.fit(X, y).transform(X)

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    170         if nc:
    171             err = f"All values missing in column(s) {nc}. Should be removed."
--> 172             raise ValueError(err)
    173         return func(d, *args, **kwargs)
    174     return wrapper

ValueError: All values missing in column(s) ['Churned']. Should be removed.

It's telling me that missing values in the column ['Churned'] should be removed. Hmm...

Let's check my training_features data again:

'Churned' in training_features.columns

True

Am I going insane? I can't figure this out. What's happening to my dataframe? And there are no missing values in Churned. It's either 1 or 0 for churn / not churned.

ValueError: Found array with 0 sample(s) (shape=(0, 88)) while a minimum of 1 is required.

Hi, I tried use this module to impute but not work.
The train data data and test data has value, not it says not data error.

ValueError Traceback (most recent call last)
in
1 # proceed the same as the previous examples
----> 2 complex_lm.fit(X_train, y_train).predict(X_test)
3 complex_lm.summary()

/anaconda3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
59 err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
60 raise TypeError(err)
---> 61 return func(d, *args, **kwargs)
62 return wrapper
63

/anaconda3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
124
125 # return func if no missingness violations detected, then return wrap
--> 126 return func(d, *args, **kwargs)
127 return wrapper
128

/anaconda3/lib/python3.7/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
171 err = f"All values missing in column(s) {nc}. Should be removed."
172 raise ValueError(err)
--> 173 return func(d, *args, **kwargs)
174 return wrapper
175

/anaconda3/lib/python3.7/site-packages/autoimpute/analysis/linear_regressor.py in fit(self, X, y)
88 # then fit the analysis models on each of the imputed datasets
89 self.models_ = self._apply_models_to_mi_data(
---> 90 self.linear_models, X, y
91 )
92

/anaconda3/lib/python3.7/site-packages/autoimpute/analysis/base_regressor.py in _apply_models_to_mi_data(self, model_dict, X, y)
248 model_type = model_dict["type"]
249 regressor = model_dict[self.model_lib]
--> 250 mi_data = self._fit_strategy_validator(X, y)
251 models = {}
252

/anaconda3/lib/python3.7/site-packages/autoimpute/analysis/base_regressor.py in _fit_strategy_validator(self, X, y)
205
206 # return the multiply imputed datasets
--> 207 return self.mi.fit_transform(X)
208
209 def _fit_model(self, model_type, regressor, X, y):

/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in fit_transform(self, X, y)
229 def fit_transform(self, X, y=None):
230 """Convenience method to fit then transform the same dataset."""
--> 231 return self.fit(X, y).transform(X)

/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in fit(self, X, y)
186 visit=self.visit
187 )
--> 188 imputer.fit(X)
189 self.statistics_[i] = imputer
190

/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/dataframe/single_imputer.py in fit(self, X, y)
182 x_ = one_hot_encode(x)
183
--> 184 imputer.fit(x_, y_)
185
186 # finally, store imputer for each column as statistics

/anaconda3/lib/python3.7/site-packages/autoimpute/imputations/series/pmm.py in fit(self, X, y)
100
101 # get predictions for the data, which will be used for "closest" vals
--> 102 y_pred = self.lm.fit(X, y).predict(X)
103 y_df = DataFrame({"y": y, "y_pred": y_pred})
104

/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/base.py in fit(self, X, y, sample_weight)
461 n_jobs_ = self.n_jobs
462 X, y = check_X_y(X, y, accept_sparse=['csr', 'csc', 'coo'],
--> 463 y_numeric=True, multi_output=True)
464
465 if sample_weight is not None and np.atleast_1d(sample_weight).ndim > 1:

/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_X_y(X, y, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, multi_output, ensure_min_samples, ensure_min_features, y_numeric, warn_on_dtype, estimator)
717 ensure_min_features=ensure_min_features,
718 warn_on_dtype=warn_on_dtype,
--> 719 estimator=estimator)
720 if multi_output:
721 y = check_array(y, 'csr', force_all_finite=True, ensure_2d=False,

/anaconda3/lib/python3.7/site-packages/sklearn/utils/validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
548 " minimum of %d is required%s."
549 % (n_samples, array.shape, ensure_min_samples,
--> 550 context))
551
552 if ensure_min_features > 0 and array.ndim == 2:

ValueError: Found array with 0 sample(s) (shape=(0, 88)) while a minimum of 1 is required.

Predictive Imputer fit_transform check_nan_columns issue

The PredictiveImputer utilizes the check_nan_columns decorator for both fit and transform methods. The fit step further uses listwise_delete to compose a model on observed data. After listwise delete runs, however, the observed DataFrame may no longer pass the check_nan_columns decorator. Such is the case when when one column is fully missing for each value in another column that is completely missing, plus additional values.

In this case, the transform method will fail even though the fit method succeeds. This is because the transformation employs the check decorator again.

This one's tricky. Techincally, it's not wrong, although the error is misleading.

Should we automatically remove columns that will get in the way of imputation?
Should we keep the error, or issue a warning?
Should we employ some sort of regularization so it won't affect the model?

Current Status of Kalman imputation?

MultipleImputer transform error

hi I want to use autoimpute package.
so I test it. but it is not worked.
how to solve it?

sample = data.sample(1000)
mi.fit(sample)
mi = MultipleImputer(return_list= True) 
imputed = mi.transform(sample)

error

Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-19-41db8ffcd4a8> in <module>
----> 1 imputed = mi.transform(sample)

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    171             err = f"All values missing in column(s) {nc}. Should be removed."
    172             raise ValueError(err)
--> 173         return func(d, *args, **kwargs)
    174     return wrapper
    175 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in transform(self, X)
    224                    for i in self.statistics_.items())
    225         if self.return_list:
--> 226             imputed = list(imputed)
    227         return imputed
    228 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/imputations/dataframe/multiple_imputer.py in <genexpr>(.0)
    222         # sequential only for now
    223         imputed = ((i[0], i[1].transform(X))
--> 224                    for i in self.statistics_.items())
    225         if self.return_list:
    226             imputed = list(imputed)

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/utils/checks.py in wrapper(d, *args, **kwargs)
    171             err = f"All values missing in column(s) {nc}. Should be removed."
    172             raise ValueError(err)
--> 173         return func(d, *args, **kwargs)
    174     return wrapper
    175 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/imputations/dataframe/single_imputer.py in transform(self, X)
    259 
    260             # perform imputation given the specified imputer and value for x_
--> 261             X.loc[imp_ix, column] = imputer.impute(x_)
    262         return X
    263 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/imputations/series/default.py in impute(self, X)
    398     def impute(self, X):
    399         """Defer transform to the DefaultBaseImputer."""
--> 400         X_ = super().impute(X)
    401         return X_

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/imputations/series/default.py in impute(self, X)
    212         # ensure that param is not none, which indicates time series column
    213         if imp:
--> 214             X_ = imp.impute(X)
    215             return X_
    216 

~/anaconda3/envs/py36/lib/python3.6/site-packages/autoimpute/imputations/series/pmm.py in impute(self, X)
    153                 sample=self.sample,
    154                 tune=self.tune,
--> 155                 init=self.init,
    156             )
    157         self.trace_ = tr

~/anaconda3/envs/py36/lib/python3.6/site-packages/pymc3/sampling.py in sample(draws, step, init, n_init, start, trace, chain_idx, chains, cores, tune, progressbar, model, random_seed, discard_tuned_samples, compute_convergence_checks, **kwargs)
    394                 start_, step = init_nuts(init=init, chains=chains, n_init=n_init,
    395                                          model=model, random_seed=random_seed,
--> 396                                          progressbar=progressbar, **kwargs)
    397                 if start is None:
    398                     start = start_

~/anaconda3/envs/py36/lib/python3.6/site-packages/pymc3/sampling.py in init_nuts(init, chains, n_init, model, random_seed, progressbar, **kwargs)
   1513             'Unknown initializer: {}.'.format(init))
   1514 
-> 1515     step = pm.NUTS(potential=potential, model=model, **kwargs)
   1516 
   1517     return start, step

~/anaconda3/envs/py36/lib/python3.6/site-packages/pymc3/step_methods/hmc/nuts.py in __init__(self, vars, max_treedepth, early_max_treedepth, **kwargs)
    150         `pm.sample` to the desired number of tuning steps.
    151         """
--> 152         super().__init__(vars, **kwargs)
    153 
    154         self.max_treedepth = max_treedepth

~/anaconda3/envs/py36/lib/python3.6/site-packages/pymc3/step_methods/hmc/base_hmc.py in __init__(self, vars, scaling, step_scale, is_cov, model, blocked, potential, dtype, Emax, target_accept, gamma, k, t0, adapt_step_size, step_rand, **theano_kwargs)
     70         vars = inputvars(vars)
     71 
---> 72         super().__init__(vars, blocked=blocked, model=model, dtype=dtype, **theano_kwargs)
     73 
     74         self.adapt_step_size = adapt_step_size

~/anaconda3/envs/py36/lib/python3.6/site-packages/pymc3/step_methods/arraystep.py in __init__(self, vars, model, blocked, dtype, **theano_kwargs)
    226 
    227         func = model.logp_dlogp_function(
--> 228             vars, dtype=dtype, **theano_kwargs)
    229 
    230         # handle edge case discovered in #2948

~/anaconda3/envs/py36/lib/python3.6/site-packages/pymc3/model.py in logp_dlogp_function(self, grad_vars, **kwargs)
    721         varnames = [var.name for var in grad_vars]
    722         extra_vars = [var for var in self.free_RVs if var.name not in varnames]
--> 723         return ValueGradFunction(self.logpt, grad_vars, extra_vars, **kwargs)
    724 
    725     @property

~/anaconda3/envs/py36/lib/python3.6/site-packages/pymc3/model.py in __init__(self, cost, grad_vars, extra_vars, dtype, casting, **kwargs)
    460 
    461         self._theano_function = theano.function(
--> 462             inputs, [self._cost_joined, grad], givens=givens, **kwargs)
    463 
    464     def set_extra_values(self, extra_vars):

**TypeError: function() got an unexpected keyword argument 'sample'**

Add Chained Equations to Multiple Imputation

Right now, the MultipleImputer creates multiple samples of the same dataset, and it imputes each one independently. This is the repeat n times logic. That being said, the MultipleImputer does not actively improve each of the imputed datasets (i.e. each imputer runs only once on each column of each sample). The ChainedEquationsImputer (TBD on implementation) would handle iterative improvements to each imputation. The psuedo-code is as follows (provided by @gjdv):

repeat n times:
  identify missingness in dataframe
  initialize an imputed dataframe by inserting e.g., mean values per column where data is missing
  while not stable (or for set k number of iterations):
    for each column with missingness:
      create a single imputer using the current column as output and the other columns as input to the model
      update the imputed dataframe with imputed values where originally data was missing

Initial plan is to implement this as a NEW SeriesImputer. May need some changes to the MultipleImputer, although that is TBD.

function( ) got an unexpected keyword argument 'sample'

Hi,
I tried SingleImputer on one of my datasets and it is failing, if there are missing values in the column that should be predicted.

And then I tried to run the example given in your tutorial:
Link: https://kearnz.github.io/autoimpute-tutorials/
(Getting the Most out of the Imputer Classes: Part III)
I tried to run SingleImputer

I'm running into the same error.
Auto-assigning NUTS sampler...
Initializing NUTS using jitter+adapt_diag...
Traceback (most recent call last):
File "C:/Users/Desktop/Test/Code/main.py", line 177, in
si_data_full = si.fit_transform(data_miss)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\imputations\dataframe\single_imputer.py", line 275, in fit_transform
return self.fit(X, y).transform(X)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\utils\checks.py", line 61, in wrapper
return func(d, *args, **kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\utils\checks.py", line 126, in wrapper
return func(d, *args, **kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\utils\checks.py", line 173, in wrapper
return func(d, *args, **kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\imputations\dataframe\single_imputer.py", line 261, in transform
X.loc[imp_ix, column] = imputer.impute(x_)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\imputations\series\default.py", line 400, in impute
X_ = super().impute(X)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\imputations\series\default.py", line 214, in impute
X_ = imp.impute(X)
File "C:\Users\Desktop\Test\venv\lib\site-packages\autoimpute\imputations\series\pmm.py", line 155, in impute
init=self.init,
File "C:\Users\Desktop\Test\venv\lib\site-packages\pymc3\sampling.py", line 396, in sample
progressbar=progressbar, **kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\pymc3\sampling.py", line 1515, in init_nuts
step = pm.NUTS(potential=potential, model=model, **kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\pymc3\step_methods\hmc\nuts.py", line 152, in init
super().init(vars, **kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\pymc3\step_methods\hmc\base_hmc.py", line 72, in init
super().init(vars, blocked=blocked, model=model, dtype=dtype, **theano_kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\pymc3\step_methods\arraystep.py", line 228, in init
vars, dtype=dtype, **theano_kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\pymc3\model.py", line 723, in logp_dlogp_function
return ValueGradFunction(self.logpt, grad_vars, extra_vars, **kwargs)
File "C:\Users\Desktop\Test\venv\lib\site-packages\pymc3\model.py", line 462, in init
inputs, [self._cost_joined, grad], givens=givens, **kwargs)
TypeError: function() got an unexpected keyword argument 'sample'

Note: I don't have C++ installed.
My warning list is:
WARNING (theano.configdefaults): g++ not available, if using conda: conda install m2w64-toolchain
C:\Users\Desktop\Test\venv\lib\site-packages\theano\configdefaults.py:560: UserWarning: DeprecationWarning: there is no c++ compiler.This is deprecated and with Theano 0.11 a c++ compiler will be mandatory
warnings.warn("DeprecationWarning: there is no c++ compiler."
WARNING (theano.configdefaults): g++ not detected ! Theano will be unable to execute optimized C-implementations (for both CPU and GPU) and will default to Python implementations. Performance will be severely degraded. To remove this warning, set Theano flags cxx to an empty string.
WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions.

books to reference

Thank you for your great talk at pydata. I am reaching out to get the link for the book you mentioned about. Also, do you mind sharing the slides as well?

column inclusion for MissingnessClassifier

Right now, using a prefix and "startswith". This does not account for columns with the same unique prefix (i.e. "height" and "height_cm". Note "height_inches" and "height_cm" would not be a problem).

Need to use a key-value pair for categorical and respective dummy columns.

md_pattern argument handling

Right now, cols=None is default. But if data not dataframe, cols are mandatory. That logic is not intuitive to the user. Need to refactor so that cols are necessary if dataframe not passed. May need to break into multiple methods.

how to use DefaultTimeSeriesImputer

i read the documentation but some things still not clear enough for me:
assuming the following input data
value quantity temperature
0 1573 13597 51
1 3996 513 nan
2 nan 11589 50

using this imputation assuming the data is sorted by any time index/column?
how can i know which algorithm is used? for predict temperature value?
let assume i use MiceImputer:

imp = MiceImputer(
n=3,
strategy={"Year": "default time","value": "pmm"},
visit="left-to-right",
return_list=True
)

imp.fit_transform(df)

so two different algorithms will use? one for predict value and another one for temperature?
thanks

ValueError: This solver needs samples of at least 2 classes in the data, but the data contains only one class: 0

My data set has 91713 row and 186 columns. WIDS2020 Kaggle trainV2.csv

I have deleted the one column that had only one class.
There were were no coluumns with all 'nulls'.

Source code below with error
Python Version 3.7 Autoimpute version 0.11.7 build pypy_0 channel pypy
dataframe information

training_features_df.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 91713 entries, 0 to 91712
Columns: 186 entries, encounter_id to apache_2_bodysystem
dtypes: float64(170), int64(8), object(8)
memory usage: 130.1+ MB

begin source code

import warnings
import numpy as np
import pandas as pd
from autoimpute.imputations import SingleImputer
warnings.filterwarnings('ignore')

helper functions used throughout this project

print_header = lambda msg: print(f"{msg}\n{'-'*len(msg)}")

helper functions used throughout this project

def A100_load_data_file(file):
return pd.read_csv(file)
#return pd.read_csv(file,nrows=10000)
training_features_df = A100_load_data_file('C:/UsersCMDS/Documents/APersonal/WomenInDataScience/WIDSDatathon2020/training_v2.csv')

def summary(df):
types = df.dtypes
counts = df.apply(lambda x: x.count())
uniques = df.apply(lambda x: [x.unique()])
nas = df.apply(lambda x: x.isnull().sum())
distincts = df.apply(lambda x: x.unique().shape[0])
missing = (df.isnull().sum() /df.shape[0]) * 100
sk = df.skew()
krt = df.kurt()
print('Data shape:', df.shape)
cols = ['Type', 'Total count', 'Null Values', 'Distinct Values', 'Missing Ratio', 'Unique Values', 'Skewness', 'Kurtosis']
dtls = pd.concat([types, counts, nas, distincts, missing, uniques, sk, krt], axis=1, sort=False)
dtls.columns = cols
return dtls

details = summary(training_features_df)

check details file and sort by distinct values. drop readmission_status because it only has one class

check detail file for columns with all nulls, none found

data = training_features_df.drop('readmission_status',axis=1)

print_header("Fit Method returning instance of the SingleImputer class")

SingleImputer().fit_transform(data)

data4=SingleImputer().fit_transform(data)
pd.display.options.max_columns = 185

end source code

########### Results

runfile('C:/Users/CMDS/Documents/APersonal/WomenInDataScience/WIDSDatathon2020/untitled2.py', wdir='C:/Users/CMDSl/Documents/APersonal/WomenInDataScience/WIDSDatathon2020')
Data shape: (91713, 186)
Fit Method returning instance of the SingleImputer class

Traceback (most recent call last):

File "C:\Users\CMDSl\Documents\APersonal\WomenInDataScience\WIDSDatathon2020\untitled2.py", line 61, in
SingleImputer().fit_transform(data)

File "C:\Users\CMDS\Anaconda3\lib\site-packages\autoimpute\imputations\dataframe\single_imputer.py", line 277, in fit_transform
return self.fit(X, y).transform(X)

File "C:\Users\CMDS\Anaconda3\lib\site-packages\autoimpute\utils\checks.py", line 61, in wrapper
return func(d, *args, **kwargs)

File "C:\Users\CMDS\Anaconda3\lib\site-packages\autoimpute\utils\checks.py", line 126, in wrapper
return func(d, *args, **kwargs)

File "C:\Users\CMDS\Anaconda3\lib\site-packages\autoimpute\utils\checks.py", line 173, in wrapper
return func(d, *args, **kwargs)

File "C:\Users\CMDS\Anaconda3\lib\site-packages\autoimpute\imputations\dataframe\single_imputer.py", line 184, in fit
imputer.fit(x_, y_)

File "C:\Users\CMDSl\Anaconda3\lib\site-packages\autoimpute\imputations\series\default.py", line 395, in fit
super().fit(X, y)

File "C:\Users\CMDSl\Anaconda3\lib\site-packages\autoimpute\imputations\series\default.py", line 189, in fit
stats = {"param": self.cat_imputer.fit(X, y),

File "C:\Users\CMDS\Anaconda3\lib\site-packages\autoimpute\imputations\series\logistic_regression.py", line 151, in fit
self.glm.fit(X, y.codes)

File "C:\Users\cCMDSl\Anaconda3\lib\site-packages\sklearn\linear_model\logistic.py", line 1563, in fit
" class: %r" % classes_[0])

ValueError: This solver needs samples of at least 2 classes in the data, but the data contains only one class: 0

Question about applying MICEImputer to new data.

Hi,

Thank you for sharing a very useful and flexible implementation !
It's interesting because I've been wanting to do Multiple Imputation in Python.

I have two questions about applying MICEImputer to new data.

Now I want to do mice like mice package in R, but I also want to apply mice to new data as frequent situations in machine learning.
I tried the following code to apply MICEImputer to new data, but I got an ValueError: Variable name p_pred already exists.. When I used binary logistic and least square instead of bayesian methods, there were no errors.

First, binary logistic and least square the same as logreg and norm in mice?
Second, can beyesian methods strategy be applied to new data?

Thank you for reading my long request and sorry for poor English.

# make experiment dataset 

import numpy as np
import pandas as pd
import warnings; warnings.simplefilter('ignore')
from autoimpute.imputations import MiceImputer

N_sample = 2000
N_missing = 1000

## make data cont & cat columns
np.random.seed(0)
cat = np.random.choice([0,1], N_sample)
cont = np.random.normal(0,1,N_sample)

df_imcomplete = pd.DataFrame(np.c_[cat,cont], columns=["cat", "cont"])

## filling with NA
np.random.seed(1)
ix = np.random.choice(np.arange(N_sample), N_missing)
iy = np.random.choice([0,1], N_missing)
for i in range(N_missing):
    df_imcomplete.iloc[ix[i], iy[i]] = np.nan
    
## split
X_train = df_imcomplete[:1000]
X_test  = df_imcomplete[1000:]


# Mice imputation
imp = MiceImputer(
    n = 2,
    seed = 2,
    strategy = {"cat":"bayesian binary logistic", "cont":"bayesian least squares"},
    return_list = True,
)

# learn & apply impute
imputer = imp.fit(X_train)
X_train_imputed = imputer.transform(X_train)
X_test_imputed = imputer.transform(X_test, )

Can't Impute Using Transform Dataset Using Fitted Model

For a dataset I am working on a large proportion of variables have missing values and thought that first using a simple method (mean/normal distribution) on a copy of the dataset then fitting a stochastic regression imputation would work.

pred = {some_large_dictionary}
imp = SingleImputer(strategy='stochastic',predictors=pred,copy=False).fit(df3)
reg_imp = imp.transform(df2)

But when I try to transform the original dataset I get the following:

ValueError                                Traceback (most recent call last)
<ipython-input-179-b8b330716e07> in <module>
     12              'total_bal_ex_mort':'stochastic','total_bc_limit':'stochastic','total_il_high_credit_limit':'stochastic'}
     13 imp = SingleImputer(strategy=strat_dict,predictors=pred,copy=False).fit(df3)
---> 14 reg_imp = imp.transform(df2)

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
     59             err = f"Neither {d_err} nor {a_err} are of type pd.DataFrame"
     60             raise TypeError(err)
---> 61         return func(d, *args, **kwargs)
     62     return wrapper
     63 

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    124 
    125         # return func if no missingness violations detected, then return wrap
--> 126         return func(d, *args, **kwargs)
    127     return wrapper
    128 

~\Anaconda3\lib\site-packages\autoimpute\utils\checks.py in wrapper(d, *args, **kwargs)
    171             err = f"All values missing in column(s) {nc}. Should be removed."
    172             raise ValueError(err)
--> 173         return func(d, *args, **kwargs)
    174     return wrapper
    175 

~\Anaconda3\lib\site-packages\autoimpute\imputations\dataframe\single_imputer.py in transform(self, X)
    259 
    260             # perform imputation given the specified imputer and value for x_
--> 261             X.loc[imp_ix, column] = imputer.impute(x_)
    262         return X
    263 

~\Anaconda3\lib\site-packages\autoimpute\imputations\series\linear_regression.py in impute(self, X)
    156         check_is_fitted(self, "statistics_")
    157         mse = self.statistics_["param"]
--> 158         preds = self.lm.predict(X)
    159 
    160         # add random draw from normal dist w/ mean squared error

~\Anaconda3\lib\site-packages\sklearn\linear_model\base.py in predict(self, X)
    219             Returns predicted values.
    220         """
--> 221         return self._decision_function(X)
    222 
    223     _preprocess_data = staticmethod(_preprocess_data)

~\Anaconda3\lib\site-packages\sklearn\linear_model\base.py in _decision_function(self, X)
    202         check_is_fitted(self, "coef_")
    203 
--> 204         X = check_array(X, accept_sparse=['csr', 'csc', 'coo'])
    205         return safe_sparse_dot(X, self.coef_.T,
    206                                dense_output=True) + self.intercept_

~\Anaconda3\lib\site-packages\sklearn\utils\validation.py in check_array(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)
    540         if force_all_finite:
    541             _assert_all_finite(array,
--> 542                                allow_nan=force_all_finite == 'allow-nan')
    543 
    544     if ensure_min_samples > 0:

~\Anaconda3\lib\site-packages\sklearn\utils\validation.py in _assert_all_finite(X, allow_nan)
     54                 not allow_nan and not np.isfinite(X).all()):
     55             type_err = 'infinity' if allow_nan else 'NaN, infinity'
---> 56             raise ValueError(msg_err.format(type_err, X.dtype))
     57     # for object dtype data, we only check for NaNs (GH-13254)
     58     elif X.dtype == np.dtype('object') and not allow_nan:

ValueError: Input contains NaN, infinity or a value too large for dtype('float64').

How do I go about solving this as the error also understandably occurs when initially trying to use fit_transform on the original dataset?

MacOS (catalina) install fails

Here is what I get. I have no idea what the issue is as xgboost is already installed.
Maybe adding autoimpute to conda would do the trick?

$ pip3 install autoimpute
Requirement already satisfied: autoimpute in ./.anaconda/Anaconda3-2019.07/lib/python3.7/site-packages/autoimpute-0.11.6-py3.7.egg (0.11.6)
Requirement already satisfied: numpy in ./.anaconda/Anaconda3-2019.07/lib/python3.7/site-packages (from autoimpute) (1.16.4)
Requirement already satisfied: scipy in ./.anaconda/Anaconda3-2019.07/lib/python3.7/site-packages (from autoimpute) (1.3.0)
Requirement already satisfied: pandas in ./.anaconda/Anaconda3-2019.07/lib/python3.7/site-packages (from autoimpute) (0.24.2)
Requirement already satisfied: statsmodels in ./.anaconda/Anaconda3-2019.07/lib/python3.7/site-packages (from autoimpute) (0.10.0)
Collecting xgboost
Using cached https://files.pythonhosted.org/packages/96/84/4e2cae6247f397f83d8adc5c2a2a0c5d7d790a14a4c7400ff6574586f589/xgboost-0.90.tar.gz
ERROR: Command errored out with exit status 1:
command: /Users/josep_ferrandiz/.anaconda/Anaconda3-2019.07/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/yr/4m9kyqd97q54cmlw31gv9gmc0000gn/T/pip-install-cterd68w/xgboost/setup.py'"'"'; file='"'"'/private/var/folders/yr/4m9kyqd97q54cmlw31gv9gmc0000gn/T/pip-install-cterd68w/xgboost/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/yr/4m9kyqd97q54cmlw31gv9gmc0000gn/T/pip-install-cterd68w/xgboost/pip-egg-info
cwd: /private/var/folders/yr/4m9kyqd97q54cmlw31gv9gmc0000gn/T/pip-install-cterd68w/xgboost/
Complete output (1134 lines):
++ pwd
+ oldpath=/private/var/folders/yr/4m9kyqd97q54cmlw31gv9gmc0000gn/T/pip-install-cterd68w/xgboost
+ cd ./xgboost/
+ echo darwin19
+ grep -q darwin
+ LIB_XGBOOST=libxgboost.dylib
+ which g++-5
+ which g++-7
+ which g++-8
/usr/local/bin/g++-8
+ export CC=gcc-8
+ CC=gcc-8
+ export CXX=g++-8
+ CXX=g++-8
+ make clean
Makefile:31: MAKE [/Applications/Xcode.app/Contents/Developer/usr/bin/make] - checked OK
rm -f -rf build build_plugin lib bin *~ /~ //*~ ///~ /.o //.o ///*.o #xgboost
rm -f -rf build_tests *.gcov tests/cpp/xgboost_test
if [ -d "R-package/src" ]; then
cd R-package/src;
rm -f -rf rabit src include dmlc-core amalgamation *.so .dll;
cd /private/var/folders/yr/4m9kyqd97q54cmlw31gv9gmc0000gn/T/pip-install-cterd68w/xgboost/xgboost;
fi
+ make lib/libxgboost.dylib -j4
Makefile:31: MAKE [/Applications/Xcode.app/Contents/Developer/usr/bin/make] - checked OK
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/c_api/c_api_error.cc -o build/c_api/c_api_error.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/c_api/c_api.cc -o build/c_api/c_api.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/logging.cc -o build/logging.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/learner.cc -o build/learner.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/common/common.o src/common/common.cc >build/common/common.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/common/common.cc -o build/common/common.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/common/hist_util.o src/common/hist_util.cc >build/common/hist_util.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/common/hist_util.cc -o build/common/hist_util.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/common/host_device_vector.o src/common/host_device_vector.cc >build/common/host_device_vector.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/common/host_device_vector.cc -o build/common/host_device_vector.o
In file included from src/common/hist_util.cc:11:
src/common/./column_matrix.h: In member function 'void xgboost::common::ColumnMatrix::Init(const xgboost::common::GHistIndexMatrix&, double)':
src/common/./column_matrix.h:83:32: warning: comparison of integer expressions of different signedness: 'xgboost::bst_uint' {aka 'unsigned int'} and 'const int32_t' {aka 'const int'} [-Wsign-compare]
for (bst_uint fid = 0; fid < nfeature; ++fid) {
~~~~^~~~~~~~~~
src/common/./column_matrix.h:122:32: warning: comparison of integer expressions of different signedness: 'xgboost::bst_uint' {aka 'unsigned int'} and 'const int32_t' {aka 'const int'} [-Wsign-compare]
for (bst_uint fid = 0; fid < nfeature; ++fid) {
~~~~^~~~~~~~~~
src/common/hist_util.cc: In member function 'void xgboost::common::GHistIndexMatrix::Init(xgboost::DMatrix, int)':
src/common/hist_util.cc:313:32: warning: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'const int32_t' {aka 'const int'} [-Wsign-compare]
for (size_t tid = 0; tid < nthread; ++tid) {
~~~~^~~~~~~~~
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/data/data.o src/data/data.cc >build/data/data.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/data/data.cc -o build/data/data.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/data/simple_csr_source.o src/data/simple_csr_source.cc >build/data/simple_csr_source.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/data/simple_csr_source.cc -o build/data/simple_csr_source.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/data/simple_dmatrix.o src/data/simple_dmatrix.cc >build/data/simple_dmatrix.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/data/simple_dmatrix.cc -o build/data/simple_dmatrix.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/data/sparse_page_dmatrix.o src/data/sparse_page_dmatrix.cc >build/data/sparse_page_dmatrix.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/data/sparse_page_dmatrix.cc -o build/data/sparse_page_dmatrix.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/data/sparse_page_raw_format.o src/data/sparse_page_raw_format.cc >build/data/sparse_page_raw_format.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/data/sparse_page_source.o src/data/sparse_page_source.cc >build/data/sparse_page_source.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/data/sparse_page_raw_format.cc -o build/data/sparse_page_raw_format.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/data/sparse_page_source.cc -o build/data/sparse_page_source.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/data/sparse_page_writer.o src/data/sparse_page_writer.cc >build/data/sparse_page_writer.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/data/sparse_page_writer.cc -o build/data/sparse_page_writer.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/gbm/gblinear.o src/gbm/gblinear.cc >build/gbm/gblinear.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/gbm/gblinear.cc -o build/gbm/gblinear.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/gbm/gbm.o src/gbm/gbm.cc >build/gbm/gbm.d
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/wait.h:110,
from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:66,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/resource.h:447:34: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int getiopolicy_np(int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/resource.h:453:39: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int setiopolicy_np(int, int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:128,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/malloc/_malloc.h:52:74: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int posix_memalign(void **__memptr, size_t __alignment, size_t __size) __OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:270:53: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void arc4random_buf(void * __buf, size_t __nbytes) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:273:46: error: expected initializer before '__OSX_AVAILABLE_STARTING'
arc4random_uniform(uint32_t __upper_bound) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:292:45: error: expected initializer before '__OSX_AVAILABLE_BUT_DEPRECATED_MSG'
int daemon(int, int) __DARWIN_1050(daemon) __OSX_AVAILABLE_BUT_DEPRECATED_MSG(__MAC_10_0, __MAC_10_5, __IPHONE_2_0, __IPHONE_2_0, "Use posix_spawn APIs instead.") __WATCHOS_PROHIBITED __TVOS_PROHIBITED;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:325:6: error: expected initializer before '__OSX_AVAILABLE_STARTING'
__OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:333:6: error: expected initializer before '__OSX_AVAILABLE_STARTING'
__OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:79,
from /usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:78,
from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stdio.h:39:56: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int renameat(int, const char *, int, const char *) __OSX_AVAILABLE_STARTING(__MAC_10_10, __IPHONE_8_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:370:67: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int dprintf(int, const char * __restrict, ...) __printflike(2, 3) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:371:79: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int vdprintf(int, const char * __restrict, __gnuc_va_list) __printflike(2, 0) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:372:123: error: expected initializer before '__OSX_AVAILABLE_STARTING'
ssize_t getdelim(char ** __restrict __linep, size_t * __restrict __linecapp, int __delimiter, FILE * __restrict __stream) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:373:105: error: expected initializer before '__OSX_AVAILABLE_STARTING'
ssize_t getline(char ** __restrict __linep, size_t * __restrict __linecapp, FILE * __restrict __stream) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:197:67: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcpcpy(wchar_t * __restrict, const wchar_t * __restrict) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:198:76: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcpncpy(wchar_t * __restrict, const wchar_t * __restrict, size_t) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:199:34: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcsdup(const wchar_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:200:54: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int wcscasecmp(const wchar_t , const wchar_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:201:65: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int wcsncasecmp(const wchar_t , const wchar_t , size_t n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:202:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
size_t wcsnlen(const wchar_t , size_t) __pure __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:215:47: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t fgetwln(FILE * __restrict, size_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/gbm/gbm.cc -o build/gbm/gbm.o
In file included from dmlc-core/include/dmlc/./base.h:206,
from dmlc-core/include/dmlc/parameter.h:25,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:234:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:244:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:249:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:255:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:262:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:268:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
In file included from dmlc-core/include/dmlc/././logging.h:21,
from dmlc-core/include/dmlc/./json.h:24,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:35:27: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int backtrace(void,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:41:44: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char backtrace_symbols(void const,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:42:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void backtrace_symbols_fd(void const,int,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from dmlc-core/include/dmlc/./json.h:24,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
dmlc-core/include/dmlc/././logging.h: In function 'std::__cxx11::string dmlc::StackTrace(size_t, size_t)':
dmlc-core/include/dmlc/././logging.h:71:17: error: 'backtrace' was not declared in this scope
int nframes = backtrace(stack.data(), static_cast(stack_size));
^~~~~~~~~
dmlc-core/include/dmlc/././logging.h:71:17: note: suggested alternative: 'StackTrace'
int nframes = backtrace(stack.data(), static_cast(stack_size));
^~~~~~~~~
StackTrace
dmlc-core/include/dmlc/././logging.h:73:17: error: 'backtrace_symbols' was not declared in this scope
char **msgs = backtrace_symbols(stack.data(), nframes);
^~~~~~~~~~~~~~~~~
dmlc-core/include/dmlc/././logging.h:73:17: note: suggested alternative: 'stacktrace_os'
char **msgs = backtrace_symbols(stack.data(), nframes);
^~~~~~~~~~~~~~~~~
stacktrace_os
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h: At global scope:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:131:62: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char *stpncpy(char *__dst, const char *__src, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:132:45: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char *strndup(const char *__s1, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:133:48: error: expected initializer before '__OSX_AVAILABLE_STARTING'
size_t strnlen(const char *__s1, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:155:94: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void *memmem(const void *__big, size_t __big_len, const void *__little, size_t __little_len) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:156:75: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern4(void *__b, const void *__pattern4, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:157:75: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern8(void *__b, const void *__pattern8, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:158:77: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern16(void *__b, const void *__pattern16, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:180,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:85:17: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int ffsl(long) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:86:23: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int ffsll(long long) __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:87:15: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int fls(int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:88:17: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int flsl(long) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:89:23: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int flsll(long long) __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from include/xgboost/./../../src/common/span.h:34,
from include/xgboost/./data.h:20,
from include/xgboost/gbm.h:18,
from src/gbm/gblinear.cc:10:
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:61:11: error: '::imaxabs' has not been declared
using ::imaxabs;
^~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:62:11: error: '::imaxdiv' has not been declared
using ::imaxdiv;
^~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:68:11: error: '::strtoimax' has not been declared
using ::strtoimax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:69:11: error: '::strtoumax' has not been declared
using ::strtoumax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:72:11: error: '::wcstoimax' has not been declared
using ::wcstoimax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:73:11: error: '::wcstoumax' has not been declared
using ::wcstoumax;
^~~~~~~~~
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp -MM -MT build/gbm/gbtree.o src/gbm/gbtree.cc >build/gbm/gbtree.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -fopenmp src/gbm/gbtree.cc -o build/gbm/gbtree.o
make: *** [build/gbm/gblinear.o] Error 1
make: *** Waiting for unfinished jobs....
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/wait.h:110,
from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:66,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/resource.h:447:34: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int getiopolicy_np(int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/resource.h:453:39: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int setiopolicy_np(int, int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:128,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/malloc/_malloc.h:52:74: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int posix_memalign(void **__memptr, size_t __alignment, size_t __size) __OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:270:53: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void arc4random_buf(void * __buf, size_t __nbytes) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:273:46: error: expected initializer before '__OSX_AVAILABLE_STARTING'
arc4random_uniform(uint32_t __upper_bound) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:292:45: error: expected initializer before '__OSX_AVAILABLE_BUT_DEPRECATED_MSG'
int daemon(int, int) __DARWIN_1050(daemon) __OSX_AVAILABLE_BUT_DEPRECATED_MSG(__MAC_10_0, __MAC_10_5, __IPHONE_2_0, __IPHONE_2_0, "Use posix_spawn APIs instead.") __WATCHOS_PROHIBITED __TVOS_PROHIBITED;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:325:6: error: expected initializer before '__OSX_AVAILABLE_STARTING'
__OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:333:6: error: expected initializer before '__OSX_AVAILABLE_STARTING'
__OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:79,
from /usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:78,
from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stdio.h:39:56: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int renameat(int, const char *, int, const char *) __OSX_AVAILABLE_STARTING(__MAC_10_10, __IPHONE_8_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gbtree.cc:8:
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:370:67: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int dprintf(int, const char * __restrict, ...) __printflike(2, 3) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:371:79: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int vdprintf(int, const char * __restrict, __gnuc_va_list) __printflike(2, 0) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gbtree.cc:8:
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:372:123: error: expected initializer before '__OSX_AVAILABLE_STARTING'
ssize_t getdelim(char ** __restrict __linep, size_t * __restrict __linecapp, int __delimiter, FILE * __restrict __stream) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:373:105: error: expected initializer before '__OSX_AVAILABLE_STARTING'
ssize_t getline(char ** __restrict __linep, size_t * __restrict __linecapp, FILE * __restrict __stream) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:197:67: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcpcpy(wchar_t * __restrict, const wchar_t * __restrict) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:198:76: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcpncpy(wchar_t * __restrict, const wchar_t * __restrict, size_t) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:199:34: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcsdup(const wchar_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:200:54: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int wcscasecmp(const wchar_t , const wchar_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:201:65: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int wcsncasecmp(const wchar_t , const wchar_t , size_t n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:202:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
size_t wcsnlen(const wchar_t , size_t) __pure __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:215:47: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t fgetwln(FILE * __restrict, size_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from dmlc-core/include/dmlc/./base.h:206,
from dmlc-core/include/dmlc/parameter.h:25,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:234:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:244:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:249:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:255:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:262:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:268:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
In file included from dmlc-core/include/dmlc/././logging.h:21,
from dmlc-core/include/dmlc/./json.h:24,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:35:27: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int backtrace(void,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:41:44: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char backtrace_symbols(void const,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:42:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void backtrace_symbols_fd(void const,int,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from dmlc-core/include/dmlc/./json.h:24,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gbtree.cc:8:
dmlc-core/include/dmlc/././logging.h: In function 'std::__cxx11::string dmlc::StackTrace(size_t, size_t)':
dmlc-core/include/dmlc/././logging.h:71:17: error: 'backtrace' was not declared in this scope
int nframes = backtrace(stack.data(), static_cast(stack_size));
^~~~~~~~~
dmlc-core/include/dmlc/././logging.h:71:17: note: suggested alternative: 'StackTrace'
int nframes = backtrace(stack.data(), static_cast(stack_size));
^~~~~~~~~
StackTrace
dmlc-core/include/dmlc/././logging.h:73:17: error: 'backtrace_symbols' was not declared in this scope
char **msgs = backtrace_symbols(stack.data(), nframes);
^~~~~~~~~~~~~~~~~
dmlc-core/include/dmlc/././logging.h:73:17: note: suggested alternative: 'stacktrace_os'
char **msgs = backtrace_symbols(stack.data(), nframes);
^~~~~~~~~~~~~~~~~
stacktrace_os
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h: At global scope:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:131:62: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char *stpncpy(char *__dst, const char *__src, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:132:45: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char *strndup(const char *__s1, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:133:48: error: expected initializer before '__OSX_AVAILABLE_STARTING'
size_t strnlen(const char *__s1, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:155:94: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void *memmem(const void *__big, size_t __big_len, const void *__little, size_t __little_len) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:156:75: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern4(void __b, const void __pattern4, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:157:75: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern8(void __b, const void __pattern8, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:158:77: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern16(void __b, const void __pattern16, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:180,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gbtree.cc:8:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:85:17: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int ffsl(long) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:86:23: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int ffsll(long long) __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:87:15: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int fls(int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:88:17: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int flsl(long) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:89:23: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int flsll(long long) __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/mach_init.h:62,
from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/mach.h:69,
from dmlc-core/include/dmlc/timer.h:19,
from src/gbm/gbtree.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/vm_page_size.h:59:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
extern vm_size_t vm_kernel_page_size __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/vm_page_size.h:60:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
extern vm_size_t vm_kernel_page_mask __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/mach/vm_page_size.h:61:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
extern int vm_kernel_page_shift __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from include/xgboost/./../../src/common/span.h:34,
from include/xgboost/./data.h:20,
from include/xgboost/gbm.h:18,
from src/gbm/gbtree.cc:11:
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:61:11: error: '::imaxabs' has not been declared
using ::imaxabs;
^~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:62:11: error: '::imaxdiv' has not been declared
using ::imaxdiv;
^~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:68:11: error: '::strtoimax' has not been declared
using ::strtoimax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:69:11: error: '::strtoumax' has not been declared
using ::strtoumax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:72:11: error: '::wcstoimax' has not been declared
using ::wcstoimax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:73:11: error: '::wcstoumax' has not been declared
using ::wcstoumax;
^~~~~~~~~
make: *** [build/gbm/gbtree.o] Error 1
+ echo -----------------------------
-----------------------------
+ echo 'Building multi-thread xgboost failed'
Building multi-thread xgboost failed
+ echo 'Start to build single-thread xgboost'
Start to build single-thread xgboost
+ make clean
Makefile:31: MAKE [/Applications/Xcode.app/Contents/Developer/usr/bin/make] - checked OK
rm -f -rf build build_plugin lib bin *~ /~ //~ ///~ /.o //.o ///.o #xgboost
rm -f -rf build_tests *.gcov tests/cpp/xgboost_test
if [ -d "R-package/src" ]; then
cd R-package/src;
rm -f -rf rabit src include dmlc-core amalgamation *.so .dll;
cd /private/var/folders/yr/4m9kyqd97q54cmlw31gv9gmc0000gn/T/pip-install-cterd68w/xgboost/xgboost;
fi
+ make lib/libxgboost.dylib -j4 USE_OPENMP=0
Makefile:31: MAKE [/Applications/Xcode.app/Contents/Developer/usr/bin/make] - checked OK
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/learner.o src/learner.cc >build/learner.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/logging.o src/logging.cc >build/logging.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/c_api/c_api.o src/c_api/c_api.cc >build/c_api/c_api.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/c_api/c_api_error.o src/c_api/c_api_error.cc >build/c_api/c_api_error.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/c_api/c_api_error.cc -o build/c_api/c_api_error.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/c_api/c_api.cc -o build/c_api/c_api.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/logging.cc -o build/logging.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/learner.cc -o build/learner.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/common/common.o src/common/common.cc >build/common/common.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/common/common.cc -o build/common/common.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/common/hist_util.o src/common/hist_util.cc >build/common/hist_util.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/common/hist_util.cc -o build/common/hist_util.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/common/host_device_vector.o src/common/host_device_vector.cc >build/common/host_device_vector.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/common/host_device_vector.cc -o build/common/host_device_vector.o
In file included from src/common/hist_util.cc:11:
src/common/./column_matrix.h: In member function 'void xgboost::common::ColumnMatrix::Init(const xgboost::common::GHistIndexMatrix&, double)':
src/common/./column_matrix.h:83:32: warning: comparison of integer expressions of different signedness: 'xgboost::bst_uint' {aka 'unsigned int'} and 'const int32_t' {aka 'const int'} [-Wsign-compare]
for (bst_uint fid = 0; fid < nfeature; ++fid) {
~~~~^~~~~~~~~~
src/common/./column_matrix.h:122:32: warning: comparison of integer expressions of different signedness: 'xgboost::bst_uint' {aka 'unsigned int'} and 'const int32_t' {aka 'const int'} [-Wsign-compare]
for (bst_uint fid = 0; fid < nfeature; ++fid) {
~~~~^~~~~~~~~~
src/common/hist_util.cc: In member function 'void xgboost::common::GHistIndexMatrix::Init(xgboost::DMatrix, int)':
src/common/hist_util.cc:313:32: warning: comparison of integer expressions of different signedness: 'size_t' {aka 'long unsigned int'} and 'const int32_t' {aka 'const int'} [-Wsign-compare]
for (size_t tid = 0; tid < nthread; ++tid) {
~~~~^~~~~~~~~
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/data/data.o src/data/data.cc >build/data/data.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/data/data.cc -o build/data/data.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/data/simple_csr_source.o src/data/simple_csr_source.cc >build/data/simple_csr_source.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/data/simple_csr_source.cc -o build/data/simple_csr_source.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/data/simple_dmatrix.o src/data/simple_dmatrix.cc >build/data/simple_dmatrix.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/data/simple_dmatrix.cc -o build/data/simple_dmatrix.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/data/sparse_page_dmatrix.o src/data/sparse_page_dmatrix.cc >build/data/sparse_page_dmatrix.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/data/sparse_page_dmatrix.cc -o build/data/sparse_page_dmatrix.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/data/sparse_page_raw_format.o src/data/sparse_page_raw_format.cc >build/data/sparse_page_raw_format.d
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/data/sparse_page_source.o src/data/sparse_page_source.cc >build/data/sparse_page_source.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/data/sparse_page_raw_format.cc -o build/data/sparse_page_raw_format.o
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/data/sparse_page_source.cc -o build/data/sparse_page_source.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/data/sparse_page_writer.o src/data/sparse_page_writer.cc >build/data/sparse_page_writer.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/data/sparse_page_writer.cc -o build/data/sparse_page_writer.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/gbm/gblinear.o src/gbm/gblinear.cc >build/gbm/gblinear.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/gbm/gblinear.cc -o build/gbm/gblinear.o
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/gbm/gbm.o src/gbm/gbm.cc >build/gbm/gbm.d
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/wait.h:110,
from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:66,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/resource.h:447:34: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int getiopolicy_np(int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/resource.h:453:39: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int setiopolicy_np(int, int, int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:128,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/malloc/_malloc.h:52:74: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int posix_memalign(void **__memptr, size_t __alignment, size_t __size) __OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:270:53: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void arc4random_buf(void * __buf, size_t __nbytes) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:273:46: error: expected initializer before '__OSX_AVAILABLE_STARTING'
arc4random_uniform(uint32_t __upper_bound) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstdlib:75,
from dmlc-core/include/dmlc/parameter.h:10,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:292:45: error: expected initializer before '__OSX_AVAILABLE_BUT_DEPRECATED_MSG'
int daemon(int, int) __DARWIN_1050(daemon) __OSX_AVAILABLE_BUT_DEPRECATED_MSG(__MAC_10_0, __MAC_10_5, __IPHONE_2_0, __IPHONE_2_0, "Use posix_spawn APIs instead.") __WATCHOS_PROHIBITED __TVOS_PROHIBITED;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:325:6: error: expected initializer before '__OSX_AVAILABLE_STARTING'
__OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdlib.h:333:6: error: expected initializer before '__OSX_AVAILABLE_STARTING'
__OSX_AVAILABLE_STARTING(__MAC_10_6, __IPHONE_3_2);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:79,
from /usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:78,
from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stdio.h:39:56: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int renameat(int, const char *, int, const char *) __OSX_AVAILABLE_STARTING(__MAC_10_10, __IPHONE_8_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:370:67: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int dprintf(int, const char * __restrict, ...) __printflike(2, 3) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:371:79: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int vdprintf(int, const char * __restrict, __gnuc_va_list) __printflike(2, 0) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:90,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:372:123: error: expected initializer before '__OSX_AVAILABLE_STARTING'
ssize_t getdelim(char ** __restrict __linep, size_t * __restrict __linecapp, int __delimiter, FILE * __restrict __stream) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/lib/gcc/8/gcc/x86_64-apple-darwin18/8.3.0/include-fixed/stdio.h:373:105: error: expected initializer before '__OSX_AVAILABLE_STARTING'
ssize_t getline(char ** __restrict __linep, size_t * __restrict __linecapp, FILE * __restrict __stream) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cwchar:44,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/postypes.h:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/iosfwd:40,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/ios:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/istream:38,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/sstream:38,
from dmlc-core/include/dmlc/parameter.h:12,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:197:67: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcpcpy(wchar_t * __restrict, const wchar_t * __restrict) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:198:76: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcpncpy(wchar_t * __restrict, const wchar_t * __restrict, size_t) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:199:34: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t *wcsdup(const wchar_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:200:54: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int wcscasecmp(const wchar_t , const wchar_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:201:65: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int wcsncasecmp(const wchar_t , const wchar_t , size_t n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:202:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
size_t wcsnlen(const wchar_t , size_t) __pure __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/wchar.h:215:47: error: expected initializer before '__OSX_AVAILABLE_STARTING'
wchar_t fgetwln(FILE * __restrict, size_t ) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/gbm/gbm.cc -o build/gbm/gbm.o
In file included from dmlc-core/include/dmlc/./base.h:206,
from dmlc-core/include/dmlc/parameter.h:25,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:234:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:244:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:249:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:255:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:262:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/inttypes.h:268:25: error: expected constructor, destructor, or type conversion before '(' token
__OSX_AVAILABLE_STARTING(__MAC_10_4, __IPHONE_2_0)
^
In file included from dmlc-core/include/dmlc/././logging.h:21,
from dmlc-core/include/dmlc/./json.h:24,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:35:27: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int backtrace(void,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:41:44: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char backtrace_symbols(void const,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/execinfo.h:42:49: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void backtrace_symbols_fd(void const,int,int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from dmlc-core/include/dmlc/./json.h:24,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
dmlc-core/include/dmlc/././logging.h: In function 'std::__cxx11::string dmlc::StackTrace(size_t, size_t)':
dmlc-core/include/dmlc/././logging.h:71:17: error: 'backtrace' was not declared in this scope
int nframes = backtrace(stack.data(), static_cast(stack_size));
^~~~~~~~~
dmlc-core/include/dmlc/././logging.h:71:17: note: suggested alternative: 'StackTrace'
int nframes = backtrace(stack.data(), static_cast(stack_size));
^~~~~~~~~
StackTrace
dmlc-core/include/dmlc/././logging.h:73:17: error: 'backtrace_symbols' was not declared in this scope
char **msgs = backtrace_symbols(stack.data(), nframes);
^~~~~~~~~~~~~~~~~
dmlc-core/include/dmlc/././logging.h:73:17: note: suggested alternative: 'stacktrace_os'
char **msgs = backtrace_symbols(stack.data(), nframes);
^~~~~~~~~~~~~~~~~
stacktrace_os
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h: At global scope:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:131:62: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char *stpncpy(char *__dst, const char *__src, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:132:45: error: expected initializer before '__OSX_AVAILABLE_STARTING'
char *strndup(const char *__s1, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:133:48: error: expected initializer before '__OSX_AVAILABLE_STARTING'
size_t strnlen(const char *__s1, size_t __n) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:155:94: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void *memmem(const void *__big, size_t __big_len, const void *__little, size_t __little_len) __OSX_AVAILABLE_STARTING(__MAC_10_7, __IPHONE_4_3);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:156:75: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern4(void *__b, const void *__pattern4, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:157:75: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern8(void *__b, const void *__pattern8, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:158:77: error: expected initializer before '__OSX_AVAILABLE_STARTING'
void memset_pattern16(void *__b, const void *__pattern16, size_t __len) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_3_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:180,
from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cstring:42,
from dmlc-core/include/dmlc/././any.h:14,
from dmlc-core/include/dmlc/./json.h:33,
from dmlc-core/include/dmlc/parameter.h:26,
from src/gbm/gblinear.cc:9:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:85:17: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int ffsl(long) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:86:23: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int ffsll(long long) __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:87:15: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int fls(int) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:88:17: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int flsl(long) __OSX_AVAILABLE_STARTING(__MAC_10_5, __IPHONE_2_0);
^~~~~~~~~~~~~~~~~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/strings.h:89:23: error: expected initializer before '__OSX_AVAILABLE_STARTING'
int flsll(long long) __OSX_AVAILABLE_STARTING(__MAC_10_9, __IPHONE_7_0);
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from include/xgboost/./../../src/common/span.h:34,
from include/xgboost/./data.h:20,
from include/xgboost/gbm.h:18,
from src/gbm/gblinear.cc:10:
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:61:11: error: '::imaxabs' has not been declared
using ::imaxabs;
^~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:62:11: error: '::imaxdiv' has not been declared
using ::imaxdiv;
^~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:68:11: error: '::strtoimax' has not been declared
using ::strtoimax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:69:11: error: '::strtoumax' has not been declared
using ::strtoumax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:72:11: error: '::wcstoimax' has not been declared
using ::wcstoimax;
^~~~~~~~~
/usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/cinttypes:73:11: error: '::wcstoumax' has not been declared
using ::wcstoumax;
^~~~~~~~~
g++-8 -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP -MM -MT build/gbm/gbtree.o src/gbm/gbtree.cc >build/gbm/gbtree.d
g++-8 -c -DDMLC_LOG_CUSTOMIZE=1 -std=c++11 -Wall -Wno-unknown-pragmas -Iinclude -Idmlc-core/include -Irabit/include -I/include -O3 -funroll-loops -msse2 -fPIC -DDISABLE_OPENMP src/gbm/gbtree.cc -o build/gbm/gbtree.o
make: *** [build/gbm/gblinear.o] Error 1
.....

TypeError: function() got an unexpected keyword argument 'sample'

Hi. I am interested in dealing with missing values and found your efforts very valuable. However, I just started by copying the code of Tutorial Part 1 from the link:
https://kearnz.github.io/autoimpute-tutorials/
and while I have been trying to run it I got the following error"
Traceback (most recent call last):
File "Imp_Autoimpute_2.py", line 61, in
data_imputed_once = SingleImputer().fit_transform(data_miss)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/imputations/dataframe/single_imputer.py", line 275, in fit_transform
return self.fit(X, y).transform(X)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/utils/checks.py", line 61, in wrapper
return func(d, *args, **kwargs)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/utils/checks.py", line 126, in wrapper
return func(d, *args, **kwargs)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/utils/checks.py", line 173, in wrapper
return func(d, *args, **kwargs)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/imputations/dataframe/single_imputer.py", line 261, in transform
X.loc[imp_ix, column] = imputer.impute(x_)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/imputations/series/default.py", line 400, in impute
X_ = super().impute(X)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/imputations/series/default.py", line 214, in impute
X_ = imp.impute(X)
File "/home/baligh/.local/lib/python3.7/site-packages/autoimpute/imputations/series/pmm.py", line 155, in impute
init=self.init,
File "/usr/pkg/lib/python3.7/site-packages/pymc3/sampling.py", line 396, in sample
progressbar=progressbar, **kwargs)
File "/usr/pkg/lib/python3.7/site-packages/pymc3/sampling.py", line 1515, in init_nuts
step = pm.NUTS(potential=potential, model=model, **kwargs)
File "/usr/pkg/lib/python3.7/site-packages/pymc3/step_methods/hmc/nuts.py", line 152, in init
super().init(vars, **kwargs)
File "/usr/pkg/lib/python3.7/site-packages/pymc3/step_methods/hmc/base_hmc.py", line 72, in init
super().init(vars, blocked=blocked, model=model, dtype=dtype, **theano_kwargs)
File "/usr/pkg/lib/python3.7/site-packages/pymc3/step_methods/arraystep.py", line 228, in init
vars, dtype=dtype, **theano_kwargs)
File "/usr/pkg/lib/python3.7/site-packages/pymc3/model.py", line 723, in logp_dlogp_function
return ValueGradFunction(self.logpt, grad_vars, extra_vars, **kwargs)
File "/usr/pkg/lib/python3.7/site-packages/pymc3/model.py", line 462, in init
inputs, [self._cost_joined, grad], givens=givens, **kwargs)
TypeError: function() got an unexpected keyword argument 'sample'

Best,
Bal.

pandas Series support

Tuples, lists, numpy arrays (and their sublasses), and DataFrames are supported, but Series are not. Series have some tricky issues in converting to numpy arrays, but they should be integrated given that DataFrames implicitly rely on Series as their columns

Multiprocessing for Multiple Imputation

Multiple Imputation requires the same imputation process to be performed across each of the m datasets, where m is the number of imputations the user wants to perform. Naturally, each imputation is independent, so there's obvious gains from running them in parallel. That being said, some of the underlying imputation methods use multiprocessing to speed up the imputation model.

For example, bayesian methods utilize multiple imputation during the MCMC phase of the imputation algorithm.

Multiple processes cannot spawn multiple processes within them. Therefore, a user cannot run the the m imputations in parallel if within each imputation, an imputation model uses multiprocessing as well.

It's not clear the best way to proceed. Sampling with a single chain dramatically slows down MCMC. But performing m imputations sequentially is slow as well. As the number of m imputations grow, it may be more efficient to run them all in parallel, with the underlying imputation models utilizing only one core.

This would require significant design consideration and testing. Enough so that it may be worth exploring alternate strategies. The best workaround right now is to deploy a SingleImputer to a dedicated linux machine, but I don't have multiple boxes available to do so and therefore cannot test this capability at this point in time.

For this project to progress, however, something needs to be done about speeding up multiple imputation, as it can drag on when the number of imputations is large and/or the dataset is large as well.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

kearnz / autoimpute Goto Github PK

autoimpute's Introduction

👋👋

autoimpute's People

Contributors

Stargazers

Watchers

Forkers

autoimpute's Issues

begin source code

helper functions used throughout this project

helper functions used throughout this project

check details file and sort by distinct values. drop readmission_status because it only has one class

check detail file for columns with all nulls, none found

end source code

runfile('C:/Users/CMDS/Documents/APersonal/WomenInDataScience/WIDSDatathon2020/untitled2.py', wdir='C:/Users/CMDSl/Documents/APersonal/WomenInDataScience/WIDSDatathon2020') Data shape: (91713, 186) Fit Method returning instance of the SingleImputer class

Recommend Projects

Recommend Topics

Recommend Org

runfile('C:/Users/CMDS/Documents/APersonal/WomenInDataScience/WIDSDatathon2020/untitled2.py', wdir='C:/Users/CMDSl/Documents/APersonal/WomenInDataScience/WIDSDatathon2020')
Data shape: (91713, 186)
Fit Method returning instance of the SingleImputer class