We are finding that the parts in the tape do not align precisely. This misalignment (particularly when there is a rotation) causes the part to be misplaced. Ultimately I would like to have the part aligned over the up camera during automatic placement. Presently, I would be happy to be able to do it manually. The new wobble routine makes it easier to make this happen. One takes the part over the up camera and makes the adjustments. Then with the part centered over the up camera one records the position in the CNC domain. That value is used in the new wobble routine to calculate the offset.
Now the question is. How to add this functionality to the UI? The process as I visualize it is to pick up a part, move it to the camera, make the adjustments and place it. There are several ways to approach this in the UI.
It is my thinking that after the adjustments we set a temporary needle offset. This temporary needle offset is only used by the "Place Here" button. Any number of actions (new pick up, etc.) would clear the state and return the "Place Here" button to normal operation.
At a minimum there is a new button to indicate that a part is in position and the new offset calculated. The terminology is a problem. I don't know if saying temporary needle offset would confuse anyone not privy to the discussions here. Maybe "Adjust Place Here" ?
Another approach would be a wizard.
Adjusting a part over the up camera
Re: Adjusting a part over the up camera
OpenPnP has the vision part, and Jason has been kind enough to show us where to start porting that to a C# program. Maybe the automatic way would be doable as well?
For manual, I would have it as another place method. The part is taken to the camera, user adjusts it to be straight and centered. Press enter when happy, the part is placed. The required correction is valid for this part only, and since it is not necessary to show to the user, it does not need a term either.
For manual, I would have it as another place method. The part is taken to the camera, user adjusts it to be straight and centered. Press enter when happy, the part is placed. The required correction is valid for this part only, and since it is not necessary to show to the user, it does not need a term either.
Re: Adjusting a part over the up camera
My €0,02:
1. Calibrated, virtual A-axis center, should be the reference for all other calibrations - as discussed in other threads.
2. Measurement of displacement and rotational errror upwards facing camera should work fully automatic.
Two approaches I can think of:
a) It could be done iteratively, by centering and rotating the part until it's perfectly aligned above up cam; similar to homing on fiducials. The necessary correction is known from CNC coordinates and would give us the necessary correction for placement.
Pro: Very precise, no Z-dependent (projection) error, camera lense distortion and errors in px-mm-ratio do not matter, lesser requirements towards camera resolution and contrast.
Con: Relatively slow.
b) Measure rotation and displacement from picture (snapshot) and directly calculate necessary correction movements for placement. This is more or less how OpenPNP works, isn't it? I have not looked at the code yet.
Pro: Faster.
Con: mm-to-px ration needs to be very accurate, Z-level of part, camera lense distortion, etc. affect accuracy.
IMHO this is the killer feature we've been waiting for since day one... It's also noted as a feature request on GitHub:
https://github.com/jkuusama/LitePlacer-DEV/issues/34
1. Calibrated, virtual A-axis center, should be the reference for all other calibrations - as discussed in other threads.
2. Measurement of displacement and rotational errror upwards facing camera should work fully automatic.
Two approaches I can think of:
a) It could be done iteratively, by centering and rotating the part until it's perfectly aligned above up cam; similar to homing on fiducials. The necessary correction is known from CNC coordinates and would give us the necessary correction for placement.
Pro: Very precise, no Z-dependent (projection) error, camera lense distortion and errors in px-mm-ratio do not matter, lesser requirements towards camera resolution and contrast.
Con: Relatively slow.
b) Measure rotation and displacement from picture (snapshot) and directly calculate necessary correction movements for placement. This is more or less how OpenPNP works, isn't it? I have not looked at the code yet.
Pro: Faster.
Con: mm-to-px ration needs to be very accurate, Z-level of part, camera lense distortion, etc. affect accuracy.
IMHO this is the killer feature we've been waiting for since day one... It's also noted as a feature request on GitHub:
https://github.com/jkuusama/LitePlacer-DEV/issues/34
Re: Adjusting a part over the up camera
I think we are talking about three different things.
At this point I was talking about a fully manual operation. Picking up a part that I know what it is, but the machine doesn't. Moving the part to the camera, making adjustments, setting them to be used by place here, moving to the spot and placing the part. I have changed test button 3 to "Adjust Place Here".
Then Juha is talking about a semi-automatic operation. In the placement of a part, it moves to the up camera and waits for the user to make adjustments and signal that they are done. There are two things that the user needs to indicate. One, that up camera adjustment be used. I think this should be on a part by part basis. Perhaps a new method in the Job Data? "Place using up camera manually"? And two when the part has been adjusted and ready to place.
The third is what Malte is talking about and is the Holy Grail, fully automatic. In this case I think a new method in the Job Data like "Place Using Up Camera Automatically" would be all the UI needs.
I think the manual and semi-automatic is kind of trivial. I already have some code that seems to work well. There are some changes I need to make. One, I need to be able to do rotations at other than 90 degree increments. This is requiring some thought. I would like to rotate the cross over a part to determine the real alignment. Then show that angle with the cross in the up camera and adjust to it. Then the part is simply moved and placed, no possible errors from rotation. The UI to do this is not so simple.
I haven't checked out how OpenPNP is doing it but I expect some kind of pattern matching. One sets up video processing for a given part and using something like edge detection creates a pattern. The pattern is stored for that part. To adjust a part that pattern gets rotated to the desired angle and the part is rotated and moved until it matches the pattern.
At this point I was talking about a fully manual operation. Picking up a part that I know what it is, but the machine doesn't. Moving the part to the camera, making adjustments, setting them to be used by place here, moving to the spot and placing the part. I have changed test button 3 to "Adjust Place Here".
Then Juha is talking about a semi-automatic operation. In the placement of a part, it moves to the up camera and waits for the user to make adjustments and signal that they are done. There are two things that the user needs to indicate. One, that up camera adjustment be used. I think this should be on a part by part basis. Perhaps a new method in the Job Data? "Place using up camera manually"? And two when the part has been adjusted and ready to place.
The third is what Malte is talking about and is the Holy Grail, fully automatic. In this case I think a new method in the Job Data like "Place Using Up Camera Automatically" would be all the UI needs.
I think the manual and semi-automatic is kind of trivial. I already have some code that seems to work well. There are some changes I need to make. One, I need to be able to do rotations at other than 90 degree increments. This is requiring some thought. I would like to rotate the cross over a part to determine the real alignment. Then show that angle with the cross in the up camera and adjust to it. Then the part is simply moved and placed, no possible errors from rotation. The UI to do this is not so simple.
I haven't checked out how OpenPNP is doing it but I expect some kind of pattern matching. One sets up video processing for a given part and using something like edge detection creates a pattern. The pattern is stored for that part. To adjust a part that pattern gets rotated to the desired angle and the part is rotated and moved until it matches the pattern.
Re: Adjusting a part over the up camera
I think the "holy grail" is what we need 
For a high-level impression of how it works in OpenPNP, watch the following videos:
One video about vision processing pipeline:
https://www.youtube.com/watch?v=iUEP0bILAU0
If I understand correctly, the algorithm is basically looking for pads and edges to determine center and alignment of part. This, it does not need to have a lib with samples of all parts to achieve this.
Another video about the placement and rotation correction:
https://www.youtube.com/watch?v=pRYQaFKhsuw
This shows - as discussed earlier - that OpenPNP measures displacement and roation first and then corrects it; for demo purposes the part is perfectly aligned above the cam in the video.
Cheers
Malte
For a high-level impression of how it works in OpenPNP, watch the following videos:
One video about vision processing pipeline:
https://www.youtube.com/watch?v=iUEP0bILAU0
If I understand correctly, the algorithm is basically looking for pads and edges to determine center and alignment of part. This, it does not need to have a lib with samples of all parts to achieve this.
Another video about the placement and rotation correction:
https://www.youtube.com/watch?v=pRYQaFKhsuw
This shows - as discussed earlier - that OpenPNP measures displacement and roation first and then corrects it; for demo purposes the part is perfectly aligned above the cam in the video.
Cheers
Malte