4ri%dZddlZddlZddlmZmZddlmZddlm Z ddl m Z m Z m Z mZmZmZmZmZddlZddlmZdd lmZmZmZmZmZmZmZmZm Z m!Z!m"Z"dd l#m$Z$m%Z%m&Z&m'Z'gd Z(ee ej)ege!fe ej)egee!ffZ*e eee e+e fgeeee e+e fffZ,d e-d e e+e fde e+e ffdZ. d-d e e+e fdede-deeedeee+deee*ee*fdeee+e efde/deee defdZ0GddZ1dedeeeeej)ej)fe%fde-d e e+e fd e-d!ee,d"e/d#e/d$e/de1fd%Z2d&eeedeefd'Z3 d.d e e+e fdede-deeeeej)ej)fe%fde-d"e/d#e/d*eee+ee+fdeee*ee*fdeee+e efd!ee,d e-deee d$e/d+e/de e+eee4e1fff d,Z5dS)/z+Library with training routines of LightGBM.N) OrderedDict defaultdict) attrgetter)Path)AnyCallableDictIterableListOptionalTupleUnion)callback) BoosterDataset LightGBMError_choose_param_value_ConfigAliases_InnerPredictor!_LGBM_BoosterEvalMethodResultType6_LGBM_BoosterEvalMethodResultWithStandardDeviationType_LGBM_CustomObjectiveFunction_LGBM_EvalFunctionResultType _log_warning)SKLEARN_INSTALLED_LGBMBaseCrossValidator_LGBMGroupKFold_LGBMStratifiedKFold)cv CVBoostertrainnum_boost_round_kwargparamsreturncfdtjdD}td|t|dkrStt |dkrSdd|D}td|ddd S) aChoose number of boosting rounds. In ``train()`` and ``cv()``, there are multiple ways to provide configuration for the number of boosting rounds to perform: * the ``num_boost_round`` keyword argument * any of the ``num_iterations`` or its aliases via the ``params`` dictionary These should be preferred in the following order (first one found wins): 1. ``num_iterations`` provided via ``params`` (because it's the main parameter name) 2. any other aliases of ``num_iterations`` provided via ``params`` 3. the ``num_boost_round`` keyword argument This function handles that choice, and issuing helpful warnings in the cases where the result might be surprising. Returns ------- params : dict Parameters, with ``"num_iterations"`` set to the preferred value and all other aliases of ``num_iterations`` removed. c*i|]}|v||Sr().0aliasr$s o/var/www/html/bestrading.cuttalo.com/services/ml-inference/venv/lib/python3.11/site-packages/lightgbm/engine.py z*_choose_num_iterations..Ms2&&&!&RW[aRaRave}RaRaRanum_iterationsmain_param_namer$ default_valuerz, c3*K|]\}}|d|VdS)=Nr()r)r*vals r+ z)_choose_num_iterations..cs4gg*%----ggggggr-zCFound conflicting values for num_iterations provided via 'params': z. LightGBM will perform up to z boosting rounds. To be confident in the maximum number of boosting rounds LightGBM will perform and to suppress this warning, modify 'params' so that only one of those is present.) rgetrlensetvaluesjoinitemsr)r#r$num_iteration_configs_provided value_strings ` r+_choose_num_iterationsr>5s0&&&&*8*<=M*N*N&&&" !(+F )**a//  3-4466 7 788A== 99gg@^@d@d@f@fgggggL Wl W W'-.>'? W W W Mr-dF train_setnum_boost_round valid_sets valid_namesfeval init_modelkeep_training_booster callbacksc  t|ts%tdt|jdt|t rRt |D]B\} } t| ts(td| dt| jdCtj|}td|d}d} t|dr |d} d|d<t|| }|d }|d krtd |d td|d}|d| d|dd} d} t|tt frt#j||} n;t|t&r&t#j|t+|jfi|} | | }nd }||| d}d}g}g}|t|tr|g}t|tr|g}t |D]\} }||ur d}||| }|||||/t9|| kr||| |d| |t;}nOt |D]0\} }|jd| t9|z 1t;|}tAj!|dd rl|"tAj#|d| |ddtd|d dd kd|D}||z }tI|tKd}tI|tKd} t'||}|r|&|tO||D]\}}|(|| |)|D]}|)n2#|)|D]}|)wxYwd |_*tW|||zD]} |D](}|tAj,||| |||zd)|-| g}|R|r(|.|/||.|0| |D](}|tAj,||| |||z|)#t@j1$r!}|j*dz|_*|j2}Yd}~nd}~wwxYwtgth|_2|D]\}} }!}"|!|j2|| <|s9|5|67|S)!aPerform the training with given parameters. Parameters ---------- params : dict Parameters for training. Values passed through ``params`` take precedence over those supplied via arguments. train_set : Dataset Data to be trained on. num_boost_round : int, optional (default=100) Number of boosting iterations. valid_sets : list of Dataset, or None, optional (default=None) List of data to be evaluated on during training. valid_names : list of str, or None, optional (default=None) Names of ``valid_sets``. feval : callable, list of callable, or None, optional (default=None) Customized evaluation function. Each evaluation function should accept two parameters: preds, eval_data, and return (eval_name, eval_result, is_higher_better) or list of such tuples. preds : numpy 1-D array or numpy 2-D array (for multi-class task) The predicted values. For multi-class task, preds are numpy 2-D array of shape = [n_samples, n_classes]. If custom objective function is used, predicted values are returned before any transformation, e.g. they are raw margin instead of probability of positive class for binary task in this case. eval_data : Dataset A ``Dataset`` to evaluate. eval_name : str The name of evaluation function (without whitespaces). eval_result : float The eval result. is_higher_better : bool Is eval result higher better, e.g. AUC is ``is_higher_better``. To ignore the default metric corresponding to the used objective, set the ``metric`` parameter to the string ``"None"`` in ``params``. init_model : str, pathlib.Path, Booster or None, optional (default=None) Filename of LightGBM model or Booster instance used for continue training. keep_training_booster : bool, optional (default=False) Whether the returned Booster will be used to keep training. If False, the returned value will be converted into _InnerPredictor before returning. This means you won't be able to use ``eval``, ``eval_train`` or ``eval_valid`` methods of the returned Booster. When your model is very large and cause the memory error, you can try to set this param to ``True`` to avoid the model conversion performed during the internal call of ``model_to_string``. You can still use _InnerPredictor as ``init_model`` for future continue training. callbacks : list of callable, or None, optional (default=None) List of callback functions that are applied at each iteration. See Callbacks in Python API for more information. Note ---- A custom objective function can be provided for the ``objective`` parameter. It should accept two parameters: preds, train_data and return (grad, hess). preds : numpy 1-D array or numpy 2-D array (for multi-class task) The predicted values. Predicted values are returned before any transformation, e.g. they are raw margin instead of probability of positive class for binary task. train_data : Dataset The training dataset. grad : numpy 1-D array or numpy 2-D array (for multi-class task) The value of the first order derivative (gradient) of the loss with respect to the elements of preds for each sample point. hess : numpy 1-D array or numpy 2-D array (for multi-class task) The value of the second order derivative (Hessian) of the loss with respect to the elements of preds for each sample point. For multi-class task, preds are numpy 2-D array of shape = [n_samples, n_classes], and grad and hess should be returned in the same format. Returns ------- booster : Booster The trained Booster model. z9train() only accepts Dataset object, train_set has type ''.z8Every item in valid_sets must be a Dataset object. Item z has type ' objectiveNr/noner#r$r.r6Number of boosting rounds must be greater than 0. Got .early_stopping_roundfirst_metric_onlyF model_filepred_parameterboosterrStrainingTvalid_orderearly_stopping_min_delta verbosityrstopping_roundsrP min_deltaverbosec4h|]}t|dd|Sbefore_iterationFgetattrr)cbs r+ ztrain.."+ f f fwrK]_d?e?e f f f fr-key)r$r@modelr$ iterationbegin_iteration end_iterationevaluation_result_listfobj)8 isinstancer TypeErrortype__name__list enumeratecopydeepcopyrcallabler> ValueErrorpopr6strrrfrom_model_filer from_boosterdictr$current_iteration_update_params_set_predictorappend set_referencer7r8__dict__ setdefaultr_should_enable_early_stoppingaddearly_stoppingsortedrset_train_data_namezip add_valid_reverse_update_paramsbest_iterationrange CallbackEnvupdateextend eval_train eval_validEarlyStopException best_scorerrmodel_from_stringmodel_to_string free_dataset)#r$r@rArBrCrDrErFrGi valid_itemrrrP predictorinit_iterationis_valid_contain_traintrain_data_namereduced_valid_setsname_valid_sets valid_data callbacks_setrfcallbacks_before_iter_setcallbacks_after_iter_setcallbacks_before_itercallbacks_after_iterrU valid_setname_valid_setrpearlyStopException dataset_name eval_namescore_s# r+r"r"msl i ) )rpTXYbTcTcTlpppqqq*d##&z22  MAzj'22 HHH*.z*:*:*CHHH ]6 " "F #F 59D{#$$%k"${ #/RX Y Y YF-.O!dRadddeee!.F $%- )*** #6>>+/I*sDk**w#3zZ`aaa J ( (w#0TXYcYjTuTuntTuTuvvv "4466 V$$33I>>>" OO j' * * &$J k3 ' ' (&-K&z22 5 5MAzY&&)-&*&1!nO  % %j&?&?&G&G&U&UV_&`&` a a a&3{+;+;a+?+?&&{1~6666&&|||4444 y)) @ @EAr K " "7AI,> ? ? ? ?I -fjj9OQR.S.STT    # &'= >"3 **%?EE+$/!"##k""     !g fm f f f,/HH"#<*WBUBUVVV!": 7@S@STTT /9=== ! 9  ' ' 8 8 8),-?)Q)Q 9 9 %I~   i 8 8 8 8 9 ((***+ / /I  , , . . . . / ((***+ / /I  , , . . . . /G>>O#C D D$$'  B B$!!$2"0?"B+/      D!!!JL  !% I&--g.@.@.G.GHHH " ) )'*<* > > > ? ?r- num_iterationstart_iterationimportance_typecg}|jD]-}|||||.||jdS)zSerialize CVBooster to dict.)rrr)rr)rrrr)rrrr models_strrUs r+_to_dictzCVBooster._to_dictsh }  G   ''"/bq(     '$:MNNNr-namecXdtdtdttffd }|S)z#Redirect methods call of CVBooster.argskwargsr%crg}jD]+}|t||i|,|S)z>Call methods with each booster, and concatenate their results.)rrrd)rrretrUrrs r+handler_functionz/CVBooster.__getattr__..handler_functionsOC= D D 177D114B6BBCCCCJr-)rr )rrrs`` r+ __getattr__zCVBooster.__getattr__sH C 3 49        r-c t|SN)vars)rs r+ __getstate__zCVBooster.__getstate__sDzzr-statecJt||dSr)rr)rrs r+ __setstate__zCVBooster.__setstate__s" T %     r-rcT|tj||S)zLoad CVBooster from a string. Parameters ---------- model_str : str Model will be loaded from this string. Returns ------- self : CVBooster Loaded CVBooster object. )rrloads)rrs r+rzCVBooster.model_from_strings%  9--... r-rsplitcTtj||||S)a\Save CVBooster to JSON string. Parameters ---------- num_iteration : int or None, optional (default=None) Index of the iteration that should be saved. If None, if the best iteration exists, it is saved; otherwise, all iterations are saved. If <= 0, all iterations are saved. start_iteration : int, optional (default=0) Start index of the iteration that should be saved. importance_type : str, optional (default="split") What type of feature importance should be saved. If "split", result contains numbers of times the feature is used in a model. If "gain", result contains total gains of splits which use the feature. Returns ------- str_repr : str JSON string representation of CVBooster. )rdumpsr)rrrrs r+rzCVBooster.model_to_strings$4z$-- XXYYYr-filenamect|d5}tj|||||dddn #1swxYwY|S)aSave CVBooster to a file as JSON text. Parameters ---------- filename : str or pathlib.Path Filename to save CVBooster. num_iteration : int or None, optional (default=None) Index of the iteration that should be saved. If None, if the best iteration exists, it is saved; otherwise, all iterations are saved. If <= 0, all iterations are saved. start_iteration : int, optional (default=0) Start index of the iteration that should be saved. importance_type : str, optional (default="split") What type of feature importance should be saved. If "split", result contains numbers of times the feature is used in a model. If "gain", result contains total gains of splits which use the feature. Returns ------- self : CVBooster Returns self. wN)rrdumpr)rrrrrrs r+ save_modelzCVBooster.save_models:(C  \D IdmmM?OTTVZ [ [ [ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ s+AA A r)Nrr)rv __module__ __qualname____doc__r rr~rrr rrintrrr rrrrrrr(r-r+r!r!bs!*2611U39-.1111$?c3h?D???? Ohsm Oc O\_ Odhilnqiqdr O O O O    #sT#Y1F(G     d38n!$sCx.!T!!!!3;$(, & ZZ}ZZ Z ZZZZ>(, &  T "  }           r-r! full_datafoldsnfoldseedfpreproc stratifiedshuffleeval_train_metricc |}|} |t|dst|dstdt|dr|} | Qt j| t j} t jtt| | } n t j | t j} | t j | || }ntfdt!jd Drt$st'd t j|t j} t jtt| | } t) } | t j | | }n |rdt$st'd t+||} | t j | |}n|r3t j|| nt j| t5| z fdtd| DfdtD}t7|}t9}|D]\}}|t=|}|t=|}|$|||\}}}n}tA||}|r|!|d|!|d|j"#||S)z2Make a n-fold list of Booster from random indices.N__iter__rzyfolds should be a generator or iterator of (train_idx, test_idx) tuples or scikit-learn splitter object with split method)dtype)repeats)Xygroupsc3HK|]}|ddvVdS)>xendcgxe_ndcg lambdarank rank_xendcg xendcg_mart xe_ndcg_martN)r6)r) obj_aliasr$s r+r5z _make_n_folds..sP   JJy" % %` a      r-rJz'scikit-learn is required for ranking cv)n_splits)rrz*scikit-learn is required for stratified cv)rr random_state)rrc*g|]}||zSr(r()r)rksteprandidxs r+ z!_make_n_folds../s&QQQ!wq1u9}-QQQr-rcng|]0tjfdtD1S)c,g|]}|k|Sr(r()r)rktest_ids r+rz,_make_n_folds...0s#'S'S'SqAQRFF FFFr-)np concatenater)r)rrrs @r+rz!_make_n_folds..0sCkkkYZ'S'S'S'S'SE%LL'S'S'STTkkkr-r"valid)$ constructnum_datahasattrAttributeError get_grouprasarrayint32repeatrr7zerosrempty get_labelanyrr6rrrrrandom RandomState permutationarangerrr!subsetrryrrrr)rrrr$rrrrrr  group_info flatted_group group_kfoldskftrain_idr train_idxtest_idxr@rtparambooster_for_foldrrrs `` @@@r+ _make_n_foldsr#s##%%I!!##H uj)) '%2I2I  D  5' " " e",,..J%Z "(CCC " %J*@*@* U U U " B B B KK"(8"4"4 8K8K8M8MVcKddE     ,/ <<      + % O#$MNNNI$7$7$9$9JJJJIeC OO&<&.`sW q q qSWSTVWQqT1Q4rwqzz**LOU26!99=M=M N q q qr-)rrrr;) r$ metric_values one_resultr metric_name metric_valueis_higher_betterrjr+s @r+_agg_cv_resultr1Ds1< L8C M!44 IS 4 4 EL+|5E-C 0L   $ $S" - - - #  % %l 3 3 3 3  4 r q q q[h[n[n[p[p q q qqr-Tmetricsreturn_cvboosterc ( t|ts%tdt|jdt j|}td|d}d}t|dr |d}d|d<t||}|d}|d krtd |d td |d}|d | d | d d}t| ttfrtj| |}n>t| t"r'tj| t'| jfi|}nd}|2t+j dD]}| |d||d<|||t1t2}t5||||| | |||  }| t7}nOt9| D]0\}}|jd|t?| z 1t7| }tAj!| d d rl|"tAj#|d || ddtd|d dd kd|D}||z }tI|tKd}tI|tKd}tM|D]7}|D]%}|tAj'|||d |d&|(|tS|*|}|D]J\}}}} }!||d|d+|||d|d+|!K |D]%}|tAj'|||d ||&#t@j,$rM}"|"j-dz|_-|j.D]}#|j-|#_-|D]}$||$d|j-||$<Yd}"~"nd}"~"wwxYw|r||d <t'|S)!aPerform the cross-validation with given parameters. Parameters ---------- params : dict Parameters for training. Values passed through ``params`` take precedence over those supplied via arguments. train_set : Dataset Data to be trained on. num_boost_round : int, optional (default=100) Number of boosting iterations. folds : generator or iterator of (train_idx, test_idx) tuples, scikit-learn splitter object or None, optional (default=None) If generator or iterator, it should yield the train and test indices for each fold. If object, it should be one of the scikit-learn splitter classes (https://scikit-learn.org/stable/modules/classes.html#splitter-classes) and have ``split`` method. This argument has highest priority over other data split arguments. nfold : int, optional (default=5) Number of folds in CV. stratified : bool, optional (default=True) Whether to perform stratified sampling. shuffle : bool, optional (default=True) Whether to shuffle before splitting data. metrics : str, list of str, or None, optional (default=None) Evaluation metrics to be monitored while CV. If not None, the metric in ``params`` will be overridden. feval : callable, list of callable, or None, optional (default=None) Customized evaluation function. Each evaluation function should accept two parameters: preds, eval_data, and return (eval_name, eval_result, is_higher_better) or list of such tuples. preds : numpy 1-D array or numpy 2-D array (for multi-class task) The predicted values. For multi-class task, preds are numpy 2-D array of shape = [n_samples, n_classes]. If custom objective function is used, predicted values are returned before any transformation, e.g. they are raw margin instead of probability of positive class for binary task in this case. eval_data : Dataset A ``Dataset`` to evaluate. eval_name : str The name of evaluation function (without whitespace). eval_result : float The eval result. is_higher_better : bool Is eval result higher better, e.g. AUC is ``is_higher_better``. To ignore the default metric corresponding to the used objective, set ``metrics`` to the string ``"None"``. init_model : str, pathlib.Path, Booster or None, optional (default=None) Filename of LightGBM model or Booster instance used for continue training. fpreproc : callable or None, optional (default=None) Preprocessing function that takes (dtrain, dtest, params) and returns transformed versions of those. seed : int, optional (default=0) Seed used to generate the folds (passed to numpy.random.seed). callbacks : list of callable, or None, optional (default=None) List of callback functions that are applied at each iteration. See Callbacks in Python API for more information. eval_train_metric : bool, optional (default=False) Whether to display the train metric in progress. The score of the metric is calculated again after each training step, so there is some impact on performance. return_cvbooster : bool, optional (default=False) Whether to return Booster models trained on each fold through ``CVBooster``. Note ---- A custom objective function can be provided for the ``objective`` parameter. It should accept two parameters: preds, train_data and return (grad, hess). preds : numpy 1-D array or numpy 2-D array (for multi-class task) The predicted values. Predicted values are returned before any transformation, e.g. they are raw margin instead of probability of positive class for binary task. train_data : Dataset The training dataset. grad : numpy 1-D array or numpy 2-D array (for multi-class task) The value of the first order derivative (gradient) of the loss with respect to the elements of preds for each sample point. hess : numpy 1-D array or numpy 2-D array (for multi-class task) The value of the second order derivative (Hessian) of the loss with respect to the elements of preds for each sample point. For multi-class task, preds are numpy 2-D array of shape = [n_samples, n_classes], and grad and hess should be returned in the same format. Returns ------- eval_results : dict History of evaluation results of each metric. The dictionary has the following format: {'valid metric1-mean': [values], 'valid metric1-stdv': [values], 'valid metric2-mean': [values], 'valid metric2-stdv': [values], ...}. If ``return_cvbooster=True``, also returns trained boosters wrapped in a ``CVBooster`` object via ``cvbooster`` key. If ``eval_train_metric=True``, also returns the train metric history. In this case, the dictionary has the following format: {'train metric1-mean': [values], 'valid metric1-mean': [values], 'train metric2-mean': [values], 'valid metric2-mean': [values], ...}. z6cv() only accepts Dataset object, train_set has type 'rIrJNr/rKrLr.rrMrNrOrPFrQrTmetric) rrrr$rrrrrrXrYrZr[rr\c4h|]}t|dd|Srarcres r+rgzcv..,rhr-rirkrq z-meanz-stdv cvbooster)/rsrrtrurvryrzrr{r>r|r}r6r~rrrrrrr$rrrrrwr#r8rxrrr7rrrrrrrrrr1rrrrr)%r$r@rArrrrr3rDrErrrGrr4rrrPr metric_aliasresultsr9rrrfrrrrresrr. metric_meanrmetric_std_devrbstrs% r+r r csh i ) )omQUV_Q`Q`Qimmmnnn ]6 " "F #F 59D{#$$%k"${ #/RX Y Y YF-.O!dRadddeee!.F $%- )*** #6>>*sDk** #3!!    J ( (#0 1<>>$G +   I y)) @ @EAr K " "7AI,> ? ? ? ?I -fjj9OQR.S.STT    # &'= >"3 **%?EE+$/!"##k""     !g fm f f f,/HH"#<*WBUBUVVV!": 7@S@STTT ? # ###'  B B$#!$%"1+/      d###Y11%8899IL Q Q EL+{A~ |88k888 9 @ @ M M M |88k888 9 @ @ P P P P *  ('%"#()&5/2      *   '9'H1'LI $ ) > >%.%="" D D$QZ(B)*B(BC EEEEE  )(  ==s6(PQ;.AQ66Q;)r?NNNNFN) r?Nr2TTNNNNrNFF)6rryr collectionsrroperatorrpathlibrtypingrrr r r r r rnumpyrrrbasicrrrrrrrrrrrcompatrrrr__all__ndarray_LGBM_CustomMetricFunctionr~_LGBM_PreprocFunctionrr>boolr"r!r#r1r'r r(r-r+rLs11 00000000NNNNNNNNNNNNNNNNNNNN                          feeeeeeeeeee   # W$ & W )* ,  ! gtCH~& '7DcN *+- 5#5tCH~5RVWZ\_W_R`5555v*.'+[_6:"'*.rr cNrrrg' r $s)$ r E4d;U6VVW X rsD'123r rX'r rrrrjVVVVVVVVrFF E(5RZ)?#@ACZZ[ \F F cN F  F ,- FFFFFFFFRrd<=>r @ArrrrD_c/3[_6:04*.#"vv cNvvv E(5RZ)?#@ACZZ[ \ v  v  vveCcN+ ,v E4d;U6VVW XvsD'123v,-v vX'vvv  #uT%[)+, ,-!vvvvvvr-