Outside handle attaches to green clip. Inside handle attaches to the lever circled in blue. Both of them operate the same metal lever, so the problem is likely to be one of these attachment points.
About 6:30 into the video below shows the metal part moving & how the handles operate it.