{"id":380,"date":"2024-03-23T06:45:34","date_gmt":"2024-03-23T06:45:34","guid":{"rendered":"http:\/\/devashree-shukla.local\/?p=380"},"modified":"2024-03-23T11:48:45","modified_gmt":"2024-03-23T11:48:45","slug":"380-2","status":"publish","type":"post","link":"http:\/\/devashree-shukla.local\/380-2\/","title":{"rendered":"Integrating Machine Learning into iOS Apps: A starter roadmap"},"content":{"rendered":"\n<h4 class=\"wp-block-heading\">Core ML<\/h4>\n\n\n\n<ul>\n<li><strong>Overview<\/strong>: Core ML is Apple&#8217;s framework for integrating machine learning models into apps. It&#8217;s optimized for on-device performance, which minimizes memory footprint and power consumption.<\/li>\n\n\n\n<li><strong>Key Concepts<\/strong>: Model integration, Prediction, Real-time processing, Model conversion (using Core ML Tools).<\/li>\n\n\n\n<li><strong>Use Cases<\/strong>: Image classification, Sentiment analysis, Text prediction, Speech recognition.<\/li>\n\n\n\n<li>For further reading: <a href=\"https:\/\/developer.apple.com\/documentation\/coreml\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/developer.apple.com\/documentation\/coreml<\/a><\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">ARKit<\/h4>\n\n\n\n<ul>\n<li><strong>Overview<\/strong>: ARKit is Apple&#8217;s framework for developing augmented reality experiences. It combines device motion tracking, camera scene capture, advanced scene processing, and display conveniences to simplify the task of building an AR experience.<\/li>\n\n\n\n<li><strong>Key Concepts<\/strong>: World Tracking, Face Tracking, Image &amp; Object Detection, Environmental understanding, Human occlusion.<\/li>\n\n\n\n<li><strong>Use Cases<\/strong>: Virtual object placement, Interactive gaming, Retail and design, and Educational tools.<\/li>\n\n\n\n<li>For further reading: <a href=\"https:\/\/developer.apple.com\/documentation\/arkit\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/developer.apple.com\/documentation\/arkit<\/a><\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">RealityKit<\/h4>\n\n\n\n<ul>\n<li><strong>Overview<\/strong>: RealityKit is a new framework by Apple designed to work with ARKit. It focuses on rendering realistic 3D and AR content with photorealistic rendering, animation, physics, and spatial audio.<\/li>\n\n\n\n<li><strong>Key Concepts<\/strong>: Anchoring system, Entity-component system, Ray-casting, Collaborative sessions.<\/li>\n\n\n\n<li><strong>Use Cases<\/strong>: Complex AR applications, Realistic simulations, Collaborative AR experiences.<\/li>\n\n\n\n<li>For further reading: <a href=\"https:\/\/developer.apple.com\/documentation\/RealityKit\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/developer.apple.com\/documentation\/RealityKit<\/a><\/li>\n<\/ul>\n\n\n\n<h4 class=\"wp-block-heading\">Vision Framework<\/h4>\n\n\n\n<ul>\n<li><strong>Overview<\/strong>: The Vision Framework uses computer vision to perform face tracking, face detection, landmarks detection, text detection, and barcode recognition.<\/li>\n\n\n\n<li><strong>Key Concepts<\/strong>: Image analysis, Object tracking, Text recognition, Barcode detection.<\/li>\n\n\n\n<li><strong>Use Cases<\/strong>: Photo tagging, Interactive text features, Retail apps, Security applications.<\/li>\n\n\n\n<li>For further reading: <a href=\"https:\/\/developer.apple.com\/documentation\/vision\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/developer.apple.com\/documentation\/vision<\/a><\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Important Core ML APIs and Concepts<\/h3>\n\n\n\n<ul>\n<li><strong>MLModel<\/strong>: The core class that represents a machine learning model in Core ML. You load a model into your app by referencing its generated class, which conforms to the <code>MLModel<\/code> protocol.<\/li>\n\n\n\n<li><strong>MLModelProvider<\/strong>: A protocol that defines methods for loading and updating models dynamically, useful for apps that need to update their models without being recompiled.<\/li>\n\n\n\n<li><strong>MLFeatureValue<\/strong>: Represents the data input and output of a model. Core ML supports several data types, including numbers, strings, images (as <code>CVPixelBuffer<\/code>), multiarrays, dictionaries, and sequences.<\/li>\n\n\n\n<li><strong>MLDictionaryFeatureProvider<\/strong>: A convenient way to provide input to a model using a dictionary where keys are the input feature names of the model.<\/li>\n\n\n\n<li><strong>MLPredictionOptions<\/strong>: Options to configure model predictions, such as specifying the preferred Metal device for running the model.<\/li>\n\n\n\n<li><strong>Vision Framework Integration<\/strong>: For tasks that involve image processing before feeding into a Core ML model, the Vision framework can be used to prepare images. This is often necessary for tasks like object detection, image classification, and more.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Project Idea: Image Classifier App<\/h3>\n\n\n\n<p>A simple yet intriguing project could be an Image Classifier iOS app. This app would use the camera to capture images in real time and classify them using a pre-trained Core ML model. For demonstration purposes, let&#8217;s use MobileNet, a lightweight model suitable for mobile applications, to classify objects into predefined categories.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Step 1: Add MobileNet to Your Project<\/h4>\n\n\n\n<p>First, you need to add a Core ML model to your project. You can download the MobileNet model from Apple&#8217;s Core ML models page or use any other model that suits your interest.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Step 2: Core ML Model Integration<\/h4>\n\n\n\n<p>Once the model is added to your project, Xcode automatically generates a class for the model. You can then use this class to make predictions.<\/p>\n\n\n\n<p>Organize your code into folders like <code>Models<\/code>, <code>ViewControllers<\/code>, <code>Views<\/code>, and <code>Helpers<\/code> for better readability.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<pre>\nimport UIKit\nimport Vision\nimport CoreML\n\nclass ImageClassifier {\n    private var model: VNCoreMLModel?\n    \n    init() {\n        do {\n            let configuration = MLModelConfiguration()\n            model = try VNCoreMLModel(for: MobileNet(configuration: configuration).model)\n        } catch {\n            print(\"Error setting up Core ML model: \\(error)\")\n        }\n    }\n    \n    func classify(image: UIImage, completion: @escaping (String) -> Void) {\n        guard let model = model, let ciImage = CIImage(image: image) else {\n            completion(\"Model or image not available\")\n            return\n        }\n        \n        let request = VNCoreMLRequest(model: model) { request, error in\n            guard let results = request.results as? [VNClassificationObservation],\n                  let topResult = results.first else {\n                completion(\"Failed to classify image.\")\n                return\n            }\n            \n            completion(\"Classification: \\(topResult.identifier), Confidence: \\(topResult.confidence)\")\n        }\n        \n        let handler = VNImageRequestHandler(ciImage: ciImage)\n        do {\n            try handler.perform([request])\n        } catch {\n            print(\"Failed to perform classification.\\n\\(error.localizedDescription)\")\n        }\n    }\n}\n<\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">References <\/h3>\n\n\n\n<ul>\n<li>CoreML models: <a href=\"https:\/\/developer.apple.com\/machine-learning\/models\/#image\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/developer.apple.com\/machine-learning\/models\/#image<\/a><\/li>\n\n\n\n<li>CoreML models: <a href=\"https:\/\/github.com\/likedan\/Awesome-CoreML-Models\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/github.com\/likedan\/Awesome-CoreML-Models<\/a><\/li>\n\n\n\n<li>Demo: <a href=\"https:\/\/github.com\/tucan9389\/awesome-ml-demos-with-ios\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/github.com\/tucan9389\/awesome-ml-demos-with-ios<\/a><\/li>\n\n\n\n<li>CreateML: <a href=\"https:\/\/developer.apple.com\/machine-learning\/create-ml\/\" target=\"_blank\" rel=\"noopener\" title=\"\">https:\/\/developer.apple.com\/machine-learning\/create-ml\/<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Core ML ARKit RealityKit Vision Framework Important Core ML APIs and Concepts Project Idea: Image Classifier App A simple yet intriguing project could be an Image Classifier iOS app. This app would use the camera to capture images in real time and classify them using a pre-trained Core ML model. For demonstration purposes, let&#8217;s use&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[24],"tags":[38,35,37,40,36,39,29],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/posts\/380"}],"collection":[{"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/comments?post=380"}],"version-history":[{"count":8,"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/posts\/380\/revisions"}],"predecessor-version":[{"id":414,"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/posts\/380\/revisions\/414"}],"wp:attachment":[{"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/media?parent=380"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/categories?post=380"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/devashree-shukla.local\/wp-json\/wp\/v2\/tags?post=380"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}