Hot Door CORE Forum
tool diameter crash - Printable Version

+- Hot Door CORE Forum (http://hotdoorcore.com/forum)
+-- Forum: All forums (http://hotdoorcore.com/forum/forumdisplay.php?fid=1)
+--- Forum: Bugs (http://hotdoorcore.com/forum/forumdisplay.php?fid=5)
+--- Thread: tool diameter crash (/showthread.php?tid=190)

tool diameter crash - Rick Johnson - 11-12-2018

I've been having trouble with a couple of tools using the "[" and "]" keys to change its diameter. I assign a diameter callback like so:

Code:
this->__penTool.setIncreaseDiameterCallback(HDI_CORE_CALLBACK(pen::Plugin, this, __penToolIncDiaCB));

It successfully calls and runs the increase or decrease callback, but after anywhere from 5-15 presses, it crashes in Xcode (haven't tried it in VC++) with the attached error, apparently related to a tool message. Is there anything I can do to help determine if this is a bug?

RE: tool diameter crash - garrett - 11-19-2018

Interesting. Can you reproduce the problem in our tool sample plugin with the latest version of hdi_core, by adding some kind of simple tool diameter callbacks? If so, please let us know which version(s) of Illustrator are affected on which platforms. Also post the code for the callbacks you added. Thanks.

RE: tool diameter crash - Rick Johnson - 11-23-2018

Hi Garrett,

I added the callbacks to the Tool sample, tested it on Mac and Windows CS6 and CC 2019, and it crashes consistently on all at hdi::core::toolMessage. Here's what Xcode showed me for its CS6 crash:

ToolSample`hdi::core::ToolMessage::ToolMessage:
0x128db91c0 <+0>: pushq %rbp
0x128db91c1 <+1>: movq %rsp, %rbp
0x128db91c4 <+4>: pushq %r15
0x128db91c6 <+6>: pushq %r14
0x128db91c8 <+8>: pushq %r13
0x128db91ca <+10>: pushq %r12
0x128db91cc <+12>: pushq %rbx
0x128db91cd <+13>: subq $0xa8, %rsp
0x128db91d4 <+20>: movq %rdi, -0xb0(%rbp)
0x128db91db <+27>: movq %rsi, -0xb8(%rbp)
0x128db91e2 <+34>: movl %edx, %r14d
0x128db91e5 <+37>: movq %rcx, %r13
0x128db91e8 <+40>: callq 0x128db6270 ; hdi::core::Message::Message()
0x128db91ed <+45>: leaq 0x58eb3c(%rip), %rdx ; vtable for


I added this at the end of toolPlugin.h

Code:
        double diam;
        void __diamUp();
        void __diamDown();

Add this to the tool definition in ToolPlugin.cpp

Code:
    this->__gearTool.setIncreaseDiameterCallback(HDI_CORE_CALLBACK(tool::Plugin, this, __diamUp));
    this->__gearTool.setDecreaseDiameterCallback(HDI_CORE_CALLBACK(tool::Plugin, this, __diamDown));
    this->diam = 100;

Then the callbacks at the end of toolPlugin.cpp

Code:
void tool::Plugin::__diamUp(){
    this->diam += 1;
}

void tool::Plugin::__diamDown(){
    this->diam -=1;
}

Having a do-nothing callback seems to be safe, but it becomes fragile once the callback actually adjusts stored values. I hope this is helpful.