If the book chapter has a separate DOI, ISBN or PMID, you can just import that via the Import references menu. If not, you may need to enter it manually. Click Import references, then Create reference manually. If you choose the book type, as soon as you enter anything into the chapter field it’s treated as a chapter.
If you want to quickly make duplicates of a reference (for different chapters that have most of the same citation details), go to the reference detail page (simply click on the reference from your All references section, or within a project), click the menu icon and select Duplicate: