[m_ke]

Hey, that's a really cool project.

We haven't tried going directly from image to calories yet, and I'm not sure that we ever will. Instead the plan is to do end-to-end portion size prediction for some of the classes. Segmentation would be cool but it's really hard to get the data for it.

By the way, plotting images with matplotlib is a pain. Try using HTML with base64 encoded images instead. Something like this should work:

  def base64image(path_or_image, prefix='data:image/jpeg;base64,'):
    s = BytesIO()
    get_pil_image(path_or_image).save(s, format='JPEG')
    return prefix + base64.b64encode(s.getvalue()).decode('utf-8')


  def show_images(paths_or_images, predictions=None, sz=200, urls=False):
    from IPython.core.display import display, HTML
    predictions = predictions if predictions is not None else []
    img_tags = map(lambda p: '''
      <div style="display: inline-block; margin: 2px; width: {sz}px; height: {sz}px; position: relative">
        <img src="{b}"
             style="max-height: 100%; max-width: 100%;
                   position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%);
                   border: {bsz}px solid rgba(255, 0, 0, {pred});"/>
      </div>
      '''.format(b=p[0] if urls else base64image(p[0]), pred=1 - p[1] if p[1] is not None else 0, sz=sz, bsz=5),
                 zip_longest(paths_or_images, predictions))
    display(HTML('<div style="text-align: center">{}<div>'.format(''.join(img_tags))))