[stratospark]

Very cool! I shared a small project to demo and explain how I used convolutional neural networks to classify food images: http://blog.stratospark.com/deep-learning-applied-food-class....

I'd be curious about the calorie detection. I'm wondering if it's using some kind of weighted sum of image segmentation proportions, or doing end-to-end deep learning.

Anyway, cool product, love to see where it goes!

[m_ke]

Hey, that's a really cool project.

We haven't tried going directly from image to calories yet, and I'm not sure that we ever will. Instead the plan is to do end-to-end portion size prediction for some of the classes. Segmentation would be cool but it's really hard to get the data for it.

By the way, plotting images with matplotlib is a pain. Try using HTML with base64 encoded images instead. Something like this should work:

  def base64image(path_or_image, prefix='data:image/jpeg;base64,'):
    s = BytesIO()
    get_pil_image(path_or_image).save(s, format='JPEG')
    return prefix + base64.b64encode(s.getvalue()).decode('utf-8')


  def show_images(paths_or_images, predictions=None, sz=200, urls=False):
    from IPython.core.display import display, HTML
    predictions = predictions if predictions is not None else []
    img_tags = map(lambda p: '''
      <div style="display: inline-block; margin: 2px; width: {sz}px; height: {sz}px; position: relative">
        <img src="{b}"
             style="max-height: 100%; max-width: 100%;
                   position: absolute; left: 50%; top: 50%; transform: translate(-50%, -50%);
                   border: {bsz}px solid rgba(255, 0, 0, {pred});"/>
      </div>
      '''.format(b=p[0] if urls else base64image(p[0]), pred=1 - p[1] if p[1] is not None else 0, sz=sz, bsz=5),
                 zip_longest(paths_or_images, predictions))
    display(HTML('<div style="text-align: center">{}<div>'.format(''.join(img_tags))))