So I finally had some free time to work on OpenMV, for those of you who haven't been following this project, OpenMV is an open-source tiny machine vision module based on an STM32F4 ARM Cortex-M4 micro and an OV9650 sensor. I started this project mainly because I find the existing cameras are either too limited for their price or too expensive if they do some image processing, so one of my main goals was to make this as cheap as possible, the total cost of this module so far is around $20 for a single board, and could go as low as $15 for 500 pieces.
I wrote a simple color tracking algorithm for the camera, which I tested using an Arduino, the Arduino sends out a few commands via the serial port to the camera telling it to capture and process a frame, it then receives back the coordinates of the object and controls the servos accordingly. I was able to process around 15FPS, which is not bad given the current naive implementation. This is a video of the camera tracking a ball:
You can find the code and schematics here in a single repository, keep in mind that I'm still working on it, and I'm also considering a new revision to fix some minor issues and add an SPI flash for storage. If you have any suggestions/feedback please feel free to leave a comment.
hg clone https://code.google.com/p/openmv-camera/Update: I've created a new repo on github, this has the most recent version:
git clone https://github.com/iabdalkader/openmv.git
Hey, I am looking into constructing one of these bad boys. I will let you know how it goes!
ReplyDeleteLet me know if you need any help, and good luck :)
DeleteYeah, So I was going through a parts order and making a BOM (I also forked the repo).
DeleteI found most of the components, the ones I was curious if I could get a part number for are: LED1 (the RGB LED), Y1 (the 12 MHz Oscillator), D1 and D2, and U10
I am forming a Bill of Materials for my order and I plan on uploading it as a csv filed to my fork of the repo, so hopefully this can help people in the future! Have you found any hardware issues yet I should be aware of?
The RGB LED is (DSE-153-008) I got it from sparkfun, I had a few laying around so I used them. The crystal oscillator is (SPK-16M-5032) 18PF +/-20PPM and it's actually 16Mhz (you can use 12 or 8 if you want but you have to make changes to the library), D1 and D2 are (BAT60AE6327) and U10 is (MMZ1608Y150B). Those can be replaced with anything if you can't find them, nothing is critical here and the bead is even optional.
DeleteAs for the hardware issues, there's a small I issue with the timer used to generate the sensor's external clock, it can't be configured to 24Mhz without changing some dividers in the library (with the defaults it can be set to 28Mhz) so I was going to use another timer (namely TIM1_CH1 on pin PE9) to get an exact 24Mhz clock.
Thank you! I will let you know how it goes. I have two of the cameras coming to me in the mail and I have been starting to familiarize myself with your library, I may implement some edge detection algorithms as a starter, along the way I will probably implement a Gaussian filter and some framework for applying a 2-D FIR filter to an image.
DeleteThat sounds awesome, I really wanted some help with the image processing code, I'm currently working on the driver trying to send bigger frames and I want to clean all those hard-coded values and maybe add the prebuilt stm library and headers...I can give you access to the github repo you can add your work there ? if you want just send me your github username.
DeleteWhat I can do for now is commit to my repo (my username is cwoodall) and push pull requests to you and you can accept them to merge the changes. I am about to get swamped, so my goal is to get a board out so when I have time again I can work on this. Thank you for being so supportive.
DeleteHow have you been testing the images?
Sure, sounds great, I test the images using a serial LCD/Arduino
DeleteI'm interested in building one of these as well - what lens are you using for the image sensor?
DeleteHi, I'm not using a separate lens, this way sensors can be easily changed, we might be doing a small run for testing soon, if you're interested, please join us in the google group
Deletehttps://groups.google.com/forum/#!forum/openmvcam
wow amazing, great job!
ReplyDeleteUnfortunately i'm not a STM user and got no experience with it.
The vision tracking on that is really awesome for the price of the project and the man hours put in to the project. Great work!
ReplyDeleteHello there! Thanks very much for the code/schematics! I'd been trying to make an embedded camera module for fun with no luck (with little time) since the Adafruit modules only output at 38400baud (which is pretty much useless). But your 15fps demo gives me hope! :D
ReplyDeleteThis has a serial port interface too so it won't send out frames any much faster, although, breaking out SPI shouldn't be that hard, also note that it can't send full frames yet, but I'm working on that .
DeleteWould you please list your development tools, jtag, loader ?
ReplyDeleteThanks
Hi, I'm using the STLINK-V2 on my Discovery board as a SWD debugger with gdb and the module has a USB connector for DFU so binaries can be uploaded via USB too.. I'm also working on a separate SWD debugger project. As for the toolchain, I'm using a gcc arm toolchain (see this post for details about that)
DeleteCool project!
ReplyDeleteI have checked the code and I can see, that you are using only 160x120 frames.
Probably there is not enough RAM to use bigger frames?
How would you store bigger JPEG images to FLASH (if it would be added?)
Keep up the good work.
Thanks,
Matej
Hi, yes there is only 112KB SRAM, just about enough to store a couple of QQVGA or QCIF frames.. I'm considering adding an SRAM chip, the STM32F4 has an external memory controller, but it's only available on the bigger packages.
DeleteIn order to keep the cost down, would it be possible to transfer the image in smaller chunks, let say 1280x8 (as far as I know, JPEG uses 8x8 blocks), and compress it to JPEG on the fly and write it to some kind of storage, or send out trough UART?
DeleteOr do you need to transfer the whole image from OV9650 in one transfer?
If I'd make some minor modifications to the code, do you accept the patches?
For example, I am using different cross-compiler so I changed Makefile to define
CROSS_COMPILE and use that in all other variables.
Thanks,
Matej
you can't send the frame fast enough to keep up with the DCMI port which will overflow eventually, and it also can't be sent directly to UART or a flash because the DMA doesn't support peripheral to peripheral transactions, but even if it does, if the reading end isn't fast enough it will overflow the fifo, and storing frames on flash will destroy it (too many write cycles ?). So basically, no, the whole frame has to be transferred to memory first, on the other hand, adding an SRAM while it will solve this issue and also allow for more algorithms that require more ram, isn't that easy either :). I guess this can be done in a whole new version, for now I think that's the best it can do. as for patches, I'm moving the repo to github send me your account and I will give you access.
DeleteAny plans on getting this mass produced through seed studio or sparkfun? It'd be awesome to be able to buy these direct! You should definitely check that option out!
ReplyDeleteI sure hope so, we're considering Tindie, but I can't do that myself really, as I'm currently located in Egypt, so I'm hoping someone would be willing to help with that, in the mean time, the schematics and source code are available for anyone who wants to take a stab at building one.
DeleteThat project is getting great. Adding a external RAM and breakout some more pins like SPI will be a must :)
ReplyDeletePS: Maybe you can run a little crowdfunding from the blog. If there are some 20 or 30 persons interest on it I think you can make them for 20$ each. If you need I can receive the boards here, in Portugal, and then send it to each one but only if we are talking about 20 or 30 persons, I haven't time to handle much more than this.
I've actually looked into adding external RAM (specifically a PSRAM which is cheaper and comes in higher capacities than SRAM) but that would not only increase the cost but the size of the board too, so for now I will see what I can do with the current hardware and it's limitations, plus I think I've found a way to send bigger frames without keeping them in RAM at all, but I haven't tested it yet. As for the crowdfunding, that sounds good but I was hoping to do a bigger batch (100-500 maybe?) and to lower the costs (shipping, customs etc..) it has to be done in the US. but thanks for offering anyway :)
DeleteIf most of the readers are from US so it is better. Maybe you can do a batch on Europe too :)
DeleteIsn't 100-500 to big for first batch? I don't know how much readers do you have anyway :p
yes, most visits come from the US, but also because the suppliers and the PCB factory are located there which would save the roundtrip.. I'm not sure how many I should make, but the bigger the order the cheaper it gets of course :)
DeleteAlmost that kind of projects make these boards on china. You have services like http://www.elecrow.com/services-c-73/prototyping-service-c-73_120/pcb-assembly-p-366.html and http://imall.iteadstudio.com/open-pcb/pcb-prototyping/prototype-pcb-assembled-pcba-upfront-consulting-service.html and http://www.seeedstudio.com/propagate/ where you can get PCB assemble for small batches like 50.
DeleteWhat is the minimum resolution the camera can output?
ReplyDeleteThe camera can output SXGA (1280x1024), VGA (640x480), CIF (352x288), QVGA
Delete(320x240), QCIF(176x144), QQVGA (160x120) and QQCIF (88x72). I was able to process 160x120@30FPS (I wasn't setting the clock correctly before so I was only processing 15FPS.
I wonder that how to set up the development environment
ReplyDeleteIs size matter for detecting colors of objects, am planning to build a color sorting machine for sunflower seed..
ReplyDelete