You can compute the exact offset of a polygon or an approximation using CGAL::offset_polygon_2() or CGAL::approximated_offset_2(), respectively. Your problem is similar. The difference is that your polygons degenerate to polylines. I haven't tried it out, but conceptually you should be able to use the functions above. If it doesn't work, let us know, and we might be able to make it work.
Remove the assertion statement(s), and try again. There are 3 different functions that can be used to compute the Minkowski sum. Perhaps one will work.
If none work compute the union of the Minkowski sums of the line segments that comprise the polyline and the polygon. Here you will have to compute the Minkowski sum of a single segment, which is a degenerate polygon, and another polygon. Again, try all 3 versions and remove the assertion statements if necessary.