#805395 predict.gbm() using single.tree=TRUE does not return the correct predictions with multinomial distribution

Package:
r-cran-gbm
Source:
r-cran-gbm
Description:
GNU R package providing Generalized Boosted Regression Models
Submitter:
David Paulsen
Date:
2024-01-11 17:39:19 UTC
Severity:
normal
Tags:
#805395#5
Date:
2015-11-17 17:29:12 UTC
From:
To:
For the bernoulli distribution model, predict.gbm(model, n.trees=c(1,2), single.tree=TRUE) returns the correct results.

For the multinomial distribution model with 3 classes, the results are incorrect. The first tree is accurate, but the results for the second tree appears to contain predictions for two different classes, and third value I cannot identify. Although the data I’m using can’t be shared, the results are clearly inaccurate as they do not appear in the first 6 trees.


pretty.gbm.tree(current_upsell_gbm, 1)
#   SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight  Prediction
# 0        0    0.99070843        1         5           6      79.811835   4585 -0.01284530
# 1        0    0.78340727        2         3           4       9.726234   1669  0.02624326
# 2       -1    0.06888412       -1        -1          -1       0.000000    233  0.06888412
# 3       -1    0.01932451       -1        -1          -1       0.000000   1436  0.01932451
# 4       -1    0.02624326       -1        -1          -1       0.000000   1669  0.02624326
# 5       -1   -0.03568803       -1        -1          -1       0.000000   2856 -0.03568803
# 6       -1   -0.01284530       -1        -1          -1       0.000000     60 -0.01284530

pretty.gbm.tree(current_upsell_gbm, 2)
#   SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight    Prediction
# 0        0  9.907084e-01        1         2           6      25.187460   4585  0.0001823204
# 1       -1 -2.174955e-02       -1        -1          -1       0.000000   1669 -0.0217495506
# 2       17  4.118500e+04        3         4           5       5.837505   2856  0.0129989496
# 3       -1  1.728153e-02       -1        -1          -1       0.000000   2426  0.0172815334
# 4       -1 -1.116279e-02       -1        -1          -1       0.000000    430 -0.0111627907
# 5       -1  1.299895e-02       -1        -1          -1       0.000000   2856  0.0129989496
# 6       -1  1.823204e-04       -1        -1          -1       0.000000     60  0.0001823204

pretty.gbm.tree(current_upsell_gbm, 3)
#   SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight   Prediction
# 0        0   0.968661668        1         2           6      16.796788   4585  0.012662983
# 1       -1  -0.006388166       -1        -1          -1       0.000000   1538 -0.006388166
# 2        4   3.500000000        3         4           5       7.753722   2987  0.022472380
# 3       -1   0.030072289       -1        -1          -1       0.000000   2075  0.030072289
# 4       -1   0.005180921       -1        -1          -1       0.000000    912  0.005180921
# 5       -1   0.022472380       -1        -1          -1       0.000000   2987  0.022472380
# 6       -1   0.012662983       -1        -1          -1       0.000000     60  0.012662983



pretty.gbm.tree(current_upsell_gbm, 4)
#   SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight  Prediction
# 0        0    0.96843894        1         5           6      80.145080   4585 -0.01039978
# 1        0    0.92423139        2         3           4       7.579153   1497  0.03221919
# 2       -1    0.04372024       -1        -1          -1       0.000000    977  0.04372024
# 3       -1    0.01061048       -1        -1          -1       0.000000    520  0.01061048
# 4       -1    0.03221919       -1        -1          -1       0.000000   1497  0.03221919
# 5       -1   -0.03153981       -1        -1          -1       0.000000   3018 -0.03153981
# 6       -1   -0.01039978       -1        -1          -1       0.000000     70 -0.01039978

pretty.gbm.tree(current_upsell_gbm, 5)
#   SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight    Prediction
# 0        0  9.907084e-01        1         2           6      22.723666   4585  0.0009275118
# 1       -1 -2.021911e-02       -1        -1          -1       0.000000   1644 -0.0202191098
# 2       15  5.216700e+04        3         4           5       6.990267   2871  0.0130365491
# 3       -1  1.779042e-02       -1        -1          -1       0.000000   2423  0.0177904212
# 4       -1 -1.267468e-02       -1        -1          -1       0.000000    448 -0.0126746834
# 5       -1  1.303655e-02       -1        -1          -1       0.000000   2871  0.0130365491
# 6       -1  9.275118e-04       -1        -1          -1       0.000000     70  0.0009275118

pretty.gbm.tree(current_upsell_gbm, 6)
#   SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight   Prediction
# 0        0  9.684389e-01        1         2           6       21.27357   4585  0.008641809
# 1       -1 -1.311138e-02       -1        -1          -1        0.00000   1497 -0.013111385
# 2       17  4.118500e+04        3         4           5        8.95109   3018  0.019431912
# 3       -1  1.425335e-02       -1        -1          -1        0.00000   2548  0.014253350
# 4       -1  4.750633e-02       -1        -1          -1        0.00000    470  0.047506331
# 5       -1  1.943191e-02       -1        -1          -1        0.00000   3018  0.019431912
# 6       -1  8.641809e-03       -1        -1          -1        0.00000     70  0.008641809


predict(current_upsell_gbm, off_test1[1,], n.trees=c(1,2))
# , , 1
#              -1           0            1
# [1,] 0.06888412 -0.02174955 -0.006388166

# , , 2
#             -1           0           1
# [1,] 0.1126044 -0.04196866 -0.01949955

0.06888412 + 0.04372024
# [1] 0.1126044
#805395#8
Date:
2015-11-17 21:23:36 UTC
From:
To:
Control: reassign -1 r-cran-gbm 2.1-1
#805395#19
Date:
2018-03-27 12:27:03 UTC
From:
To:
Hello Harry,

the R packaging team is maintaining gbm in Debian.  A user has filed a
bug report against version 2.1.1 which I would like to bring to your
attention.  Please have a look here where the problem is explicitly
described:

https://bugs.debian.org/805395

Could you please comment on this?  I admit I do not fully understand
what exactly is incorrect here and what to expect.  May be some test
case would help to confirm the integrity of the code.

Kind regards

       Andreas.


Control: tags -1 help
Control: forwarded -1 Harry Southworth <harry.southworth@gmail.com>
#805395#24
Date:
2018-03-27 15:08:15 UTC
From:
To:
Hello Harry,

That's correct.  The original maintainer seems to have lost interest and
I'm now trying to polish the list of bugs in any R package in Debian.

Thats the currently packaged version of gbm in Debian but I have no
sign that the issue has changed (bug reporter is in CC).

Puh, that's relaxing for me since I do not understanding it as well. ;-)

That's a very helpful response.  I subscribed issue #21 and will wait
for a better answer.

      Andreas.

Control: forwarded -1 https://github.com/gbm-developers/gbm/issues/21
#805395#29
Date:
2018-03-27 15:35:47 UTC
From:
To:
Just had a message from Brandon Greenwell saying he hopes to look into it
after end of term. He's likely your best bet.
Harry

On 27 Mar 2018 4:08 pm, "Andreas Tille" <andreas@an3as.eu> wrote:

Hello Harry,

That's correct.  The original maintainer seems to have lost interest and
I'm now trying to polish the list of bugs in any R package in Debian.

Thats the currently packaged version of gbm in Debian but I have no
sign that the issue has changed (bug reporter is in CC).

Puh, that's relaxing for me since I do not understanding it as well. ;-)

That's a very helpful response.  I subscribed issue #21 and will wait
for a better answer.

      Andreas.

Control: forwarded -1 https://github.com/gbm-developers/gbm/issues/21
#805395#34
Date:
2018-03-27 16:47:47 UTC
From:
To:
Hello,

I clearly could have provided better documentation.

There was an issue I believe in the predict function when single.tree was
set to TRUE.

The first 6 trees of the model are printed out to show the the first two
trees for each of the three classes.

The first set of predictions are shown for using 1 and 2 trees.

The maths show that the manual addition of values from each of the trees
match the first set of predictions.

The second set of predictions do not match the expected result from each
tree.

Example:
search document for 0.1126044 shows that the value of a two-tree prediction
matches arithmetically, but the addend to 0.1126044 is not found in results
for single.tree prediction.

search document for -0.04196866 shows that this value is found
in predictions for two-tree sum and single.tree = TRUE result, and switches
class position from prediction set 1 to prediction set1 (single.tree).

David
#805395#39
Date:
2018-03-28 06:15:33 UTC
From:
To:
Hi David,

thanks for your quick response.

I need to admit I'm lacking the background to understand your example
but I read your mail that you agree that version 2.1.1 is buggy.  The
bug does not contain any input data to reproduce the issue for version
2.1.3.  Would you be able to reproduce it or may be it is even fixed
in the latest version?  If not could you imagine a fix or do you think
it can be fixed in the upcoming gbm3?  If yes to the last question is
there any estimation when gbm3 will be available from CRAN?

Kind regards

      Andreas
#805395#46
Date:
2020-01-05 13:05:09 UTC
From:
To:
Hi David,

would you mind running your data against the latest version of r-cran-gbm
(2.1.5-1) to verify whether the problem persists?

Kind regards

      Andreas.